Electronic apparatus and control method thereof

ABSTRACT

Disclosed is an electronic apparatus. The electronic apparatus includes: a camera; a first microphone; a second microphone; and a processor configured to: control the electronic apparatus to obtain first sound data through the first microphone, control the electronic apparatus to obtain second sound data through the second microphone, identify an object corresponding to the first sound data and the second sound data in image data obtained through the camera, obtain position information of the identified object from the image data, and change volume information of at least one of the first sound data or the second sound data based on the obtained position information.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2020-0004485, filed on Jan. 13,2020, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated by reference herein in its entirety.

BACKGROUND Field

The disclosure relates to an electronic apparatus and a control methodthereof, and for example, to an electronic apparatus that changes sounddata based on object position information obtained from image data, anda control method thereof.

Description of Related Art

Image data and sound data may be obtained at the same time to record avideo. The obtained sound data may be stereo sound data. The sound datamay be obtained through at least two microphones. For convenience ofexplanation, a case of recording a video of a situation in which aspeaker utters a voice is assumed.

A camera captures an image of the speaker, and the voice uttered by thespeaker may be recorded through at least two microphones at the sametime. The voice recorded in stereo and the image of the speaker may notmatch each other depending on positions of the camera and the at leasttwo microphones.

For example, based on a speaker standing beside an electronic apparatus,the speaker stands in front of a camera from the viewpoint of the cameradisposed beside the electronic apparatus. Because the speaker stands infront of the camera, the speaker displayed on a display may bepositioned at the center of a screen. However, because the speaker doesnot stand in front of the electronic apparatus, volumes of a voicerecorded through at least two microphones disposed, for example, on theleft and right sides of the electronic apparatus, respectively may bedifferent.

From the viewpoint of a person who watches the video, the speaker isdisplayed on the center of the screen, but the voice recorded in stereomay be unbalanced left and right sides. As a distance between theelectronic apparatus and the speaker is increased, a relative ratio of adistance between one of the at least two microphones and the speaker toa distance between the other one and the speaker is increased.Therefore, the problem that the recorded voice is unbalanced left andright sides may become severe.

Particularly, the problem that a recorded sound is unbalanced left andright sides may be more prominent in a case of recording a sound atclose range like autonomous sensory meridian response (ASMR).

SUMMARY

Embodiments of the disclosure address the above disadvantages and otherdisadvantages not described above.

Embodiments of the disclosure provide an electronic apparatus thatchanges volume information of stereo voice data in consideration ofobject position information obtained from image data, and a controlmethod thereof.

According to an example embodiment of the disclosure, an electronicapparatus includes: a camera; a first microphone; a second microphone;and a processor configured to: control the electronic apparatus toobtain first sound data through the first microphone, control theelectronic apparatus to obtain second sound data through the secondmicrophone, identify an object corresponding to the first sound data andthe second sound data in image data obtained through the camera, obtainposition information of the identified object from the image data, andchange volume information of at least one of the first sound data or thesecond sound data based on the obtained position information.

The first microphone and the second microphone may be spaced apart fromeach other, and a first distance between a position of the camera and aposition of the first microphone, and a second distance between theposition of the camera and a position of the second microphone may bedifferent from each other.

The processor may be configured to identify, as the object correspondingto the first sound data and the second sound data, a subject who uttersthe first sound data and the second sound data in the image data.

The processor may be configured to obtain sound ratio informationbetween the volume information of the first sound data and the volumeinformation of the second sound data, and change the volume informationof at least one of the first sound data or the second sound data basedon the sound ratio information and the position information of theidentified object.

The processor may be configured to obtain weighted value information forchanging the volume information of at least one of the first sound dataor the second sound data based on the sound ratio information and theposition information of the identified object, and change the volumeinformation of at least one of the first sound data or the second sounddata based on the obtained weighted value information.

The processor may be configured to obtain the weighted value informationfor changing the volume information of at least one of the first sounddata or the second sound data based on the sound ratio information, anddistance ratio information between a first distance from one side of theimage data to the identified object, and a second distance from theother side of the image data to the identified object, and the otherside of the image data may be opposite to the one side of the imagedata.

The processor may be configured to obtain the weighted value informationby multiplying the distance ratio information and the sound ratioinformation, and change the volume information of the first sound dataor the second sound data by multiplying the volume information of thefirst sound data or the second sound data by the weighted valueinformation.

The processor may change the volume information by multiplying thevolume of one of the first sound data or the second sound data by theweighted value information, the one of the first sound data or thesecond sound data having a higher volume.

The electronic apparatus may further include a display, wherein theprocessor may be configured to control the display to display a userinterface (UI) including the changed volume information of the firstsound data or the second sound data.

The processor may be configured to obtain distance information betweenthe identified object and the electronic apparatus based on the imagedata, and change the volume information of at least one of the firstsound data or the second sound data based on the obtained positioninformation based on the obtained distance information being less than athreshold distance.

The processor may be configured to obtain, based on first and secondobjects being identified in the image data, first position informationof the identified first object and second position information of theidentified second object from the image data, change volume informationof at least one of first sound data or second sound data correspondingto the first object based on the first position information, and changevolume information of at least one of third sound data or fourth sounddata corresponding to the second object based on the second positioninformation.

A method of controlling an electronic apparatus according to an exampleembodiment includes: obtaining image data through a camera; obtainingfirst sound data through a first microphone and obtaining second sounddata through a second microphone; identifying an object corresponding tothe first sound data and the second sound data in the obtained imagedata; obtaining position information of the identified object from theimage data; and changing volume information of at least one of the firstsound data or the second sound data based on the obtained positioninformation.

The first microphone and the second microphone may be spaced apart fromeach other, and a first distance between a position of the camera and aposition of the first microphone, and a second distance between theposition of the camera and a position of the second microphone may bedifferent from each other.

In the identifying of the object, a subject who utters the first sounddata and the second sound data may be identified as the objectcorresponding to the first sound data and the second sound data in theimage data.

The method may include obtaining sound ratio information between thevolume information of the first sound data and the volume information ofthe second sound data, wherein in the changing of the volumeinformation, the volume information of at least one of the first sounddata or the second sound data may be changed based on the sound ratioinformation and the position information of the identified object.

The method may further include obtaining weighted value information forchanging the volume information of at least one of the first sound dataor the second sound data based on the sound ratio information and theposition information of the identified object, wherein in the changingof the volume information, the volume information of at least one of thefirst sound data or the second sound data may be changed based on theobtained weighted value information.

In the obtaining of the weighted value information, the weighted valueinformation for changing the volume information of at least one of thefirst sound data or the second sound data may be obtained based on thesound ratio information, and distance ratio information between a firstdistance from one side of the image data to the identified object, and asecond distance from the other side of the image data to the identifiedobject, and the other side of the image data may be opposite to the oneside of the image data.

In the changing of the volume information, the weighted valueinformation may be obtained by multiplying the distance ratioinformation and the sound ratio information, and the volume informationof the first sound data or the second sound data may be changed bymultiplying the volume information of the first sound data or the secondsound data by the weighted value information.

In the changing of the volume information, the volume information may bechanged by multiplying the volume of one of the first sound data or thesecond sound data by the weighted value information, the one of thefirst sound data or the second sound data having a higher volume.

The method may further include displaying a UI including the changedvolume information of the first sound data or the second sound data.

In the changing of the volume information, distance information betweena user corresponding to the identified object and the electronicapparatus may be obtained based on the image data, and the volumeinformation of at least one of the first sound data or the second sounddata may be changed based on the obtained position information based onthe obtained distance information being less than a threshold distance.

The method may further include obtaining, based on first and secondobjects being identified in the image data, first position informationof the identified first object and second position information of theidentified second object from the image data, wherein in the changing ofthe volume information, volume information of at least one of firstsound data or second sound data corresponding to the first object may bechanged based on the first position information, and volume informationof at least one of third sound data or fourth sound data correspondingto the second object may be changed based on the second positioninformation.

Additional and/or other aspects and advantages of the disclosure will beset forth in part in the description which follows and, in part, will beapparent from the description, or may be learned by practice of thedisclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing detailed description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 is a diagram illustrating an example video recording operationaccording to an embodiment;

FIG. 2 is a block diagram illustrating an example electronic apparatusaccording to an embodiment;

FIG. 3 is a block diagram illustrating an example configuration of theelectronic apparatus of FIG. 2 according to an embodiment;

FIG. 4 is a flowchart illustrating an example video recording operationaccording to an embodiment;

FIG. 5 is a flowchart illustrating an example recording operation ofFIG. 4 according to an embodiment;

FIG. 6 is a diagram illustrating an example operation of obtainingobject position information according to an embodiment;

FIG. 7 is a diagram illustrating an example of a change of sound dataaccording to an embodiment;

FIG. 8 is a diagram illustrating an example of a change of sound dataaccording to an embodiment;

FIG. 9 is a diagram illustrating an example of a change of sound dataaccording to an embodiment;

FIG. 10 is a flowchart an example operation of the electronic apparatusaccording to an embodiment;

FIG. 11 is a diagram illustrating an example recording operationaccording to an embodiment;

FIG. 12 is a diagram illustrating an example recording operationaccording to an embodiment;

FIG. 13 is a diagram illustrating an example recording operationaccording to an embodiment;

FIG. 14 is a diagram illustrating an example recording operationaccording to an embodiment;

FIG. 15 is a diagram illustrating an example user interface (UI) usedfor a control operation of the electronic apparatus according to anembodiment;

FIG. 16 is a diagram illustrating an example UI used for an operation ofconverting sound data according to an embodiment;

FIG. 17 is a diagram illustrating an example operation of convertingsound data according to a left-right reversal mode according to anembodiment;

FIG. 18 is a diagram illustrating an example operation of convertingsound data according to a position of a camera according to anembodiment;

FIG. 19 is a diagram illustrating an example operation of convertingsound data based on distance information between the electronicapparatus and an object according to an embodiment;

FIG. 20 is a diagram illustrating an example sound data change operationof FIG. 19 according an embodiment;

FIG. 21 is a diagram illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 22 is a flowchart illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 23 is a diagram illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 24 is a flowchart illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 25 is a diagram illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 26 is a flowchart illustrating an example operation of recording aplurality of objects according to an embodiment;

FIG. 27 is a flowchart illustrating an example operation considering theleft-right reversal mode according to an embodiment;

FIG. 28A is a diagram illustrating example image data applied with aleft-right reversal function according to an embodiment;

FIG. 28B is a diagram illustrating example image data applied with theleft-right reversal function according to an embodiment; and

FIG. 29 is a flowchart illustrating an example method of operating anelectronic apparatus according to an embodiment.

DETAILED DESCRIPTION

Hereinafter, the disclosure will be described in greater detail withreference to the accompanying drawings.

General terms that are currently widely used may be used in describingthe various example embodiments of the disclosure in consideration offunctions in the disclosure, but may be changed depending on theintention of those skilled in the art or a judicial precedent, theemergence of a new technique, and the like. In addition, in a specificcase, terms may be arbitrarily chosen. In this case, the meaning of suchterms will be based on a corresponding description portion of thedisclosure. Therefore, the terms used in embodiments of the disclosureshould be defined based on the meaning of the terms and the contentsthroughout the disclosure rather than simple names of the terms.

In the disclosure, an expression “have”, “may have”, “include”, “mayinclude”, or the like, indicates existence of a corresponding feature(for example, a numerical value, a function, an operation, or acomponent such as a part), and does not exclude existence of anadditional feature.

An expression “at least one of A or/and B” should be understood asindicating “A or B”, or “A and B”.

Expressions “first”, “second”, or the like, used in the disclosure mayindicate various components regardless of a sequence and/or importanceof the components, may be used simply to distinguish one component fromthe other components, and do not limit the corresponding components.

When it is mentioned that any component (for example, a first component)is (operatively or communicatively) coupled with/to or is connected toanother component (for example, a second component), it is to beunderstood that any component may be directly coupled to anothercomponent or may be coupled to another component through the othercomponent (for example, a third component).

Singular forms used herein are intended to include plural forms unlesscontext explicitly indicates otherwise. It will be further understoodthat terms “include” or “formed of” used in the disclosure specify thepresence of features, numerals, steps, operations, components, parts, orcombinations thereof mentioned in the disclosure, but do not precludethe presence or addition of one or more other features, numerals, steps,operations, components, parts, or combinations thereof.

In the disclosure, a “module”' or a “-er/or” may perform at least onefunction or operation, and be implemented by hardware or software or beimplemented by a combination of hardware and software. In addition, aplurality of “modules” or a plurality of “-ers/ors” may be integrated inat least one module and be implemented by at least one processor (notillustrated) except for a “module” or a “-er/or” that needs to beimplemented by specific hardware.

In the disclosure, a term “user” may be a person that uses an electronicapparatus or an apparatus (for example, artificial intelligence (AI)electronic apparatus) that uses an electronic apparatus.

Hereinafter, example embodiments of the disclosure will be described ingreater detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating an example video recording operationaccording to an embodiment.

Referring to FIG. 1, an electronic apparatus 100 including a camera 110,a first microphone 121, and a second microphone 122 may perform a videorecording operation.

The camera 110 may refer, for example, to a front camera. The electronicapparatus 100 may capture an image of a speaker 10 using the camera 110.It is assumed that a speaker is positioned in front of a camera. Theelectronic apparatus 100 may display image data obtained through thecamera 110 on a display of the electronic apparatus 100. An object 15corresponding to the speaker 10 may be included in the image data anddisplayed on the display. The speaker 10 is positioned in front of thecamera 110, and thus the object 15 corresponding to the speaker 10displayed on the display may be positioned at the center of the display.Although FIG. 1 illustrates a case of capturing the image of the speaker10 using the front camera, the description of FIG. 1 may be applied in asimilar manner to a case of capturing an image of the speaker 10 using arear camera.

A voice uttered by the speaker 10 may be recorded simultaneously throughthe first microphone 121 and the second microphone 122. It is assumedthat the first microphone 121 is disposed at a position close to thecamera 110, and the second microphone 122 is disposed at a position at adistance apart from the camera 110. Because the speaker 10 utters avoice near the first microphone 121, the volume of sound data recordedthrough the first microphone 121 may, for example, be higher than thatof sound data recorded through the second microphone 122.

The sound data obtained through the first microphone 121 may be used asleft stereo sound data, and the sound data obtained through the secondmicrophone 122 may be used as right stereo sound data.

FIG. 1 illustrates a case in which the first microphone 121 and thesecond microphone 122 protrude toward the outside of the electronicapparatus 100. The protruding of the first and second microphones 121,122 is illustrative and used to describe positions where the firstmicrophone 121 and the second microphone 122 are disposed. In actualimplementation, however, the first microphone 121 and the secondmicrophone 122 may not protrude toward the outside of the electronicapparatus 100. For example, a small hole may be formed in an outerportion of the electronic apparatus 100, and a microphone may bedisposed in the hole.

When viewing a video recorded by the electronic apparatus 100 it may bedifficult to determine at which position the speaker 10 actually stands,and may determine the position of the speaker 10 based only on an imagedisplayed on the display. The object 15 corresponding to the speaker 10in the video is displayed on the center of the display, but the volumeof the sound data obtained through the first microphone 121 may behigher than the volume of the sound data obtained through the secondmicrophone 122. The person who watches the video may expect that thestereo sound data is balanced left and right sides, considering theobject 15 corresponding to the speaker 10 displayed on the center.However, the left-side volume of the stereo sound data is different fromthe right-side volume of the stereo sound data, the person who watchesthe recorded video may determine that the stereo sound data isunbalanced.

FIG. 2 is a block diagram illustrating an example electronic apparatusaccording to an embodiment.

Referring to FIG. 2, the electronic apparatus 100 may include the camera110, the first microphone 121, the second microphone 122, and aprocessor (e.g., including processing circuitry) 130.

The electronic apparatus 100 according to various embodiments mayinclude at least one of, for example, a smartphone, a tablet personalcomputer (PC), a mobile phone, a desktop PC, a laptop PC, a personaldigital assistant (PDA), a portable multimedia player (PMP), an MP3player, or the like, but is not limited thereto. In some embodiments,the electronic apparatus 100 may include at least one of, for example, atelevision, a digital versatile disc (DVD) player, a media box (forexample, Samsung HomeSync™, Apple TV™, or Google TV™), or the like, butis not limited thereto.

Further, the electronic apparatus 100 may be any of various electronicdevices that include a camera and a microphone and may record a video.

The camera 110 may refer, for example, to a component for generating acaptured image obtained by capturing an image of a subject. The capturedimage may include both a video and a still image.

The camera 110 may obtain an image of at least one external device, andmay be implemented by a camera, a lens, an infrared sensor, or the like.

The camera 110 may include a lens and an image sensor. Examples of thelens include, for example, and without limitation, a general-purposelens, a wide-angle lens, and a zoom lens, and the type of lens may bedetermined depending on the type, characteristic, use environment of theelectronic apparatus 100. As the image sensor, a complementary metaloxide semiconductor (CMOS) image sensor, a charge coupled device (CCD)image sensor, or the like, may be used.

The camera 110 may output incident light as an image signal. Forexample, the camera 110 may include a lens, a pixel, and ananalog-to-digital (AD) converter. The lens may collect light of asubject to form an optical image on an image capturing region, and thepixel may output light entering through the lens as an analog imagesignal. Further, the AD converter may convert the analog image signalinto a digital image signal, and output the digital image signal. Forexample, the camera 110 may be disposed to capture an image of an areain front of the electronic apparatus 100, and generate a captured imageby capturing an image of a user present in front of the electronicapparatus 100.

Although a case that the electronic apparatus 100 according to theembodiment of the disclosure includes one camera 110 has been described,a plurality of image capturers may be arranged in actual implementation.The electronic apparatus 100 may include a plurality of image capturers,and may identify a posture of the head of the user by combining imagesreceived through the plurality of image capturers. Using a plurality ofcameras may be more effective in identifying a distance between asubject and the electronic apparatus or a posture of the head of theuser, because it is possible to more precisely analyze athree-dimensional motion in a case of using a plurality of cameras, ascompared with a case of using one camera.

The electronic apparatus 100 may further include the microphones 121 and122. The microphone may include, for example, a component for receivinga user's voice or other sounds and converting the user's voice or othersounds into audio data.

The microphones 121 and 122 may receive a user's voice in an activatedmode. For example, the microphone may be formed integrally with theelectronic apparatus 100 at an upper side, a front surface, a sidesurface, or the like, of the electronic apparatus 100. The microphonemay include various components such as a microphone collecting a user'svoice in an analog form, an amplifier circuit amplifying the collecteduser's voice, an A/D conversion circuit sampling the amplified user'svoice and converting the user's voice into a digital signal, and afilter circuit removing a noise component from the digital signalobtained by the conversion.

The microphone may include the first microphone 122 and the secondmicrophone 112. The first microphone 121 and the second microphone 122have been described above with reference to FIG. 1, and thus anoverlapping description thereof may not be repeated here.

The first microphone 121 and the second microphone 122 may be spacedapart from each other, and a first distance between a position of thecamera 110 and a position of the first microphone 121, and a seconddistance between the position of the camera 110 and a position of thesecond microphone 122 may be different from each other.

For example, the first microphone 121 and the second microphone 122 maybe separately arranged in a left region and a right region,respectively, as illustrated in FIG. 1. For example, the firstmicrophone 121 may be positioned in a first region, and the secondmicrophone 122 may be positioned in a second region. The camera 110 maybe positioned in any one of the first region or the second region.Therefore, a distance between the camera 110 and the first microphone121 may be different from that between the camera 110 and the secondmicrophone 122.

The processor 130 may include various processing circuitry and mayperform an overall control operation of the electronic apparatus. Theprocessor 130 functions to control an overall operation of theelectronic apparatus.

The processor 130 may be implemented, for example, and withoutlimitation, by a digital signal processor (DSP) processing a digitalsignal, a microprocessor, a time controller (TCON), or the like.However, the disclosure is not limited thereto, and the processor 130may include, for example, and without limitation, one or more of acentral processing unit (CPU), a dedicated processor, a micro controllerunit (MCU), a micro processing unit (MPU), a controller, an applicationprocessor (AP), a graphics-processing unit (GPU), a communicationprocessor (CP), an ARM processor, or the like, or may be defined bythese terms. In addition, the processor 130 may be implemented by asystem-on-chip (SoC) or large scale integration (LSI) in which aprocessing algorithm is embedded, or may be implemented in a fieldprogrammable gate array (FPGA) form. The processor 130 may performvarious functions by executing computer executable instructions storedin a memory.

The processor 130 may perform a control (e.g., control the electronicapparatus 10) to obtain first sound data through the first microphone121. The processor 130 may obtain second sound data through the secondmicrophone 122. Further, the processor 130 may obtain image data throughthe camera 110. The first sound data and the second sound data may besound data recorded at the same time. The sound data may refer, forexample, to audio data, and may include voice data and non-voice data.The first sound data may be used as left stereo sound data, and thesecond sound data may be used as right stereo sound data. The firstsound data may be used as right stereo sound data, and the second sounddata may be used as left stereo sound data. In other words, theelectronic apparatus 100 may generate stereo sound data using the firstsound data and the second sound data, and may obtain one video contentby combining the obtained image data and the generated stereo sounddata.

The processor 130 may identify an object corresponding to the firstsound data and the second sound data in the image data obtained throughthe camera 110, obtain position information of the identified objectfrom the image data, and change volume information of at least one ofthe first sound data or the second sound data based on the obtainedposition information.

The object corresponding to the first sound data and the second sounddata may refer, for example, to an object corresponding to a subjectthat generates a sound source of the first sound data and the secondsound data. For example, it is assumed that a speaker utters a voice,and the voice of the speaker is recorded through the first microphone121 and the second microphone 122 at the same time. The first sound datamay be sound data recorded through the first microphone 121, and thesecond sound data may be sound data recorded through the secondmicrophone 122. Further, the object corresponding to the first sounddata and the second sound data may refer, for example, to an objectcorresponding to the speaker. The object corresponding to the speakermay refer, for example, to an image of the speaker in the image dataobtained through the camera 110. The processor 130 may use an objectrecognition artificial intelligence module to identify the object. Theobject recognition artificial intelligence module may be directly storedin the electronic apparatus 100, or may be stored in a separate externalserver in some implementation examples. A relationship between thespeaker and the object may, for example, be described with the speaker10 and the object 15 corresponding to the speaker 10 in FIG. 1.

The position information of the identified object in the image data mayrefer, for example, to coordinate information of the object. Thecoordinate information may refer, for example, to information indicatinga position where the object is to be displayed based on a specificreference point. The image data may be configured with a pixel and apixel value. The object may refer, for example, to a region having aplurality of pixels and a plurality of pixel values, and the positioninformation of the object may refer, for example, to at least onerepresentative coordinate value. Further, the position information mayinclude information on coordinates at which the object is positioned inthe image data. For example, the position information may includeleft-side distance information and right-side distance information. Theleft-side distance information may refer, for example, to a distancefrom a point where the object is positioned in the image data to a leftaxis, and the left axis may refer, for example, to a vertical axiscorresponding to the leftmost side of the image data. The right-sidedistance information may refer, for example, to a distance from a pointwhere the object is positioned in the image data to a right axis, andthe right axis may refer, for example, to a vertical axis correspondingto the rightmost side of the image data. The vertical axes correspondingto the leftmost side and the rightmost side of the image data,respectively, may be replaced with vertical axes corresponding to theleftmost side and the rightmost side of the screen of the electronicapparatus. The left-side distance information (first distanceinformation) and the right-side distance information (second distanceinformation) will be described in greater detail below with reference toFIG. 6.

The processor 130 may identify, as the object corresponding to the firstsound data and the second sound data, a user (or subject) who utters thefirst sound data and the second sound data in the image data.

The image data may include an object that makes a sound. The object thatmakes a sound may be a person or a thing. For example, the person objectthat makes a sound may be a speaker who utters a voice. Further, thething object that makes a sound may refer, for example, to variousobjects such as a stream that makes a murmuring sound and a food thatmakes a frying sound.

The processor 130 may identify a plurality of objects in the image data,and may selectively identify only an object that makes a sound among theplurality of objects. Further, the processor 130 may identify a personobject that utters a voice. For example, based on the first sound dataand the second sound data being voice data, the processor 130 mayidentify, as the object corresponding to the first sound data and thesecond sound data, a person object (or user object) that utters voicedata.

The processor 130 may identify a mouth shape object, and determinewhether the identified mouth shape object is changed in real time tothereby identify a user who utters a voice. Based on the mouth shapeobject being changed in real time, the processor 130 may determine thata person object including the mouth shape object is the object thatmakes a sound.

The processor 130 may obtain sound ratio information between the volumeinformation of the first sound data and the volume information of thesecond sound data, and change the volume information of at least one ofthe first sound data or the second sound data based on the sound ratioinformation and the position information of the identified object.

The processor 130 may obtain the sound ratio information based on theobject position information being within a predetermined range. Thepredetermined range may refer, for example, to the center of the imagedata or displayed screen. The processor 130 may guide the object to makea sound at the center of the displayed screen to change the sound data.

Positioning the object corresponding to the speaker at the center of thedisplayed screen may refer, for example, to the speaker who makes asound standing in front of the camera 110. In this case, the sound madeby the speaker may be obtained through the first microphone 121 and thesecond microphone 122, and each obtained sound data may be used asreference. A description thereof will be provided in greater detailbelow with reference to FIG. 15.

Changing the volume information of the sound data may refer, forexample, to the processor 130 changing the volume of the sound data andgenerating a video based on the changed volume. Although a case that theelectronic apparatus 100 according to the disclosure changes the volumeof the sound data has been described, sound pressure, amplitude,frequency, phase offset, etc., may be changed, in addition to the volumeof the sound data.

Volume information of sound data may refer, for example, to informationcorresponding to the volume of a sound.

Sound ratio information may refer, for example, to a volume ratiobetween the first sound data and the second sound data. For example,based on the volume of the first sound data being PL, and the volume ofthe second sound data is PR, the sound ratio information may refer, forexample, to PL:PR. Further, the sound ratio information may includePR/PL or PL/PR. Although a case that the sound ratio informationincludes a value indicating a relationship between the volume of thefirst sound data and the volume of the second sound data, the soundratio information may include “1/PL” and “1/PR” in some implementationexamples. The sound ratio information may refer, for example, to “1/PL”and “1/PR” of FIGS. 7 and 8, or may refer, for example, to “PR/PL” or“PL/PR” of FIG. 9.

Changing output volume of the sound data based on the object positioninformation may refer, for example, to the output volume of the sounddata being changed based on a position where the object is positioned inthe image data.

For example, the processor 130 may obtain weighted value information forchanging the volume information of at least one of the first sound dataor the second sound data based on the sound ratio information and theposition information of the identified object, and change the volumeinformation of at least one of the first sound data or the second sounddata based on the obtained weighted value information.

The weighted value information may refer, for example, to a value bywhich the volume of the sound data is multiplied. The sound data may beclassified into pre-change sound data and post-change sound data, andthe processor 130 may obtain the post-change sound data by multiplyingthe volume of the pre-change sound data by a weighted value.

According to an embodiment, the processor 130 may apply the weightedvalue to both the first sound data and the second sound data. Forexample, the processor 130 may apply the weighted value to both thestereo sound data, and first weighted value information applied to thefirst sound data and second weighted value information applied to thesecond sound data may be different from each other. An embodiment inwhich the weighted value is applied to both the first sound data and thesecond sound data will be described in greater detail below withreference to FIGS. 7 and 8.

According to an embodiment, the processor 130 may apply the weightedvalue to only one of the first sound data or the second sound data.First weighted value information applied to the first sound data andsecond weighted value information applied to the second sound data maybe different from each other. A description of this embodiment will beprovided in greater detail below with reference to FIGS. 9 and 10.

The processor 130 may obtain weighted value information for changing thevolume information of at least one of the first sound data or the secondsound data based on the sound ratio information, and distance ratioinformation between a first distance from one side of the image data tothe identified object, and a second distance from the other side of theimage data to the identified object, and the other side of the imagedata may be opposite to the one side of the image data.

The one side of the image data may refer, for example, to a leftvertical axis corresponding to the leftmost side of the image data, andmay refer, for example, to a vertical axis 610 in FIG. 6.

The first distance from the one side of the image data to the identifiedobject may refer, for example, to a distance from the left vertical axiscorresponding to the leftmost side of the image data to a vertical axison which the object is positioned, and may refer, for example, to firstdistance information 601 in FIG. 6.

The other side of the image data may refer, for example, to a rightvertical axis corresponding to the rightmost side of the image data, andmay refer, for example, to a vertical axis 620 in FIG. 6.

The second distance from the other side of the image data to theidentified object may refer, for example, to a distance from the rightvertical axis corresponding to the rightmost side of the image data tothe vertical axis on which the object is positioned, and may refer, forexample, to second distance information 602 in FIG. 6.

The distance ratio information may refer, for example, to a ratiobetween the first distance and the second distance. For example, it isassumed by way of non-limiting example that the first distance is DL andthe second distance is DR. The distance ratio information may refer, forexample, to a reciprocal relationship of actual distances. For example,a ratio between the first distance and the second distance may be DR:DL.

The distance ratio information may include DR/DL or DL/DR. Although acase that the distance ratio information includes a value indicating arelationship between the first distance and the second distance, thedistance ratio information may include “1/DL” and “1/DR” in someimplementation examples. The distance ratio information may refer, forexample, to “1/DL” and “1/DR” of FIGS. 7 and 8, or may refer, forexample, to “DR/DL” or “DL/DR” of FIG. 9. Although an example in whichthe distance ratio information simply may refer, for example, to areciprocal of a distance has been described in the disclosure, thedistance ratio information may refer, for example, to a reciprocal ofthe square of a distance in actual implementation. For example, thedistance ratio information may refer, for example, to “1/(DL)²”,“1/(DR)²”, “(DR/DL)²”, or “(DL/DR)²”.

The processor 130 may obtain the weighted value information bymultiplying the distance ratio information and the sound ratioinformation, and change the volume information of the first sound dataor the second sound data by multiplying the volume of the first sounddata or the second sound data by the weighted value information.

The weighted value information may include first weighted valueinformation and second weighted value information. The first weightedvalue information may include a weighted value corresponding to thefirst sound data, and the second weighted value information may includea weight value corresponding to the second sound data. The firstweighted value information may be different from the second weightedvalue information.

According to an embodiment, the sound data may be changed by applyingthe first weighted value information to the first sound data, andapplying the second weighted value information to the second sound data.The processor 130 may change the first sound data and the second sounddata at the same time, and a description thereof will be provided ingreater detail below with reference to FIGS. 7 and 8.

According to an embodiment, the sound data may be changed by applyingthe first weighted value information to the first sound data, orapplying the second weighted value information to the second sound data.The processor 130 may change only one of the first sound data or thesecond sound data, and a description thereof will be provided in greaterdetail below with reference to FIGS. 9 and 10.

The processor 130 may change the volume information by multiplying thevolume of one of the first sound data or the second sound data by theweighted value information, the one sound data having a higher volume.The processor 130 may perform a sound data change operation on sounddata having a higher volume. The reason why only the sound data having ahigher volume is changed is that sound quality hardly deteriorates incase of changing sound data having a higher volume to sound data havinga lower volume. The sound quality may deteriorate in case of changingsound data having a lower volume to sound data having a higher volume.Therefore, the processor 130 may change sound data having a highervolume by comparing the volume of the first sound data and the volume ofthe second sound data. Weighted value information to be applied to thesound data may vary depending on whether to change the first sound dataor the second sound data. A description will be provided in greaterdetail below with reference to FIG. 10.

The electronic apparatus 100 may further include a display 140 (refer toFIG. 3), and the processor 130 may control the display 140 to displaythe volume information of the changed sound data. The processor 130 maycontrol the display 140 to display a user interface (UI) including thechanged volume information of the first sound data or the second sounddata. The processor 130 may obtain distance information between a usercorresponding to the identified object and the electronic apparatus 100based on the image data, and may change the volume information of atleast one of the first sound data or the second sound data based on theobtained position information based on the obtained distance informationbeing less than a threshold distance. Based on the obtained distanceinformation being equal to or greater than the threshold distance, theprocessor 130 need not perform the sound data change operation. This isbecause a difference in volume between the first sound data and thesecond sound data is not large based on the obtained distanceinformation being equal to or greater than the threshold distance, andthus an imbalance between the image data and the sound data may not besignificant. An example operation based on the distance informationbetween a user corresponding to the identified object and the electronicapparatus 100 will be described in greater detail below with referenceto FIG. 19.

The processor 130 may obtain, based on first and second objects beingidentified in the image data, first position information of theidentified first object and second position information of theidentified second object from the image data, change volume informationof at least one of first sound data (recorded through the firstmicrophone 121) or second sound data (recorded through the secondmicrophone 122) corresponding to the first object based on the firstposition information, and change volume information of at least one ofthird sound data (recorded through the first microphone 121) or fourthsound data (recorded through the second microphone 122) corresponding tothe second object based on the second position information.

According to an embodiment, based on a plurality of objects that maymake a sound being identified and one of the plurality of objectscurrently making a sound, the processor 130 may change sound data basedon position information of the object that makes a sound among theplurality of objects. A description thereof will be provided in greaterdetail below with reference to FIGS. 21 and 22.

According to an embodiment, based on a plurality of objects makingsounds, the processor 130 may obtain position information of each of theplurality of objects, and separate (or obtain) sound data correspondingto each of the plurality of objects from sound data. The processor 130may change each separated sound data based on the position informationof each of the plurality of objects. A description thereof will beprovided in greater detail below with reference to FIGS. 23 and 24.

According to an embodiment, based on sound data including both voicedata and non-voice data, the processor 130 may separate the voice dataand perform the sound data change operation on only the voice data. Incase an object corresponding to the non-voice data moves, the processor130 may perform the sound data change operation on the object, similarlyto a case of an object corresponding to the voice data. In case theobject corresponding to the non-voice data does not move, the processor130 need not perform the sound data change operation. An additionaloperation for the voice data and non-voice data will be described ingreater detail below with reference to FIGS. 25 and 26.

According to an embodiment, the processor 130 may determine whether theimage data is obtained through the front camera, and determine whether aleft-right reversal function is applied. A description thereof will beprovided in greater detail below with reference to FIGS. 27, 28A and28B.

An embodiment in which the processor 130 considers both the sound ratioinformation and the object position information has been describedabove. However, in some implementation examples, the processor 130 maychange volume information of sound data by considering only at least oneof the sound ratio information or the object position information.

The sound data change operation according to the disclosure may beapplied in case of performing recording at close range. Close range mayrefer, for example, to a distance of about 1 m. In case of performingrecording within a distance of about 1 m, the volume of the first sounddata may be different from the volume of the second sound data by about6 dB to 10 dB. Further, comparing the volume of sound, the volume of thefirst sound data may be different from the volume of the second sounddata by about two or three times.

The volume of sound may be expressed by sound intensity or soundpressure. The sound intensity may refer, for example, to acoustic energytransferred to a unit area per unit time, and the unit of the soundintensity may be w/cm². The sound pressure may be related to a changeamount of pressure generated at a specific position, and the unit of thesound pressure may be dyne/cm² or pascal (Pa).

In expressing the volume of sound, in case the sound intensity isexpressed using the decibel (dB), the volume of sound may be expressedas dB sound intensity level (dB SIL) or dB sound power level (dB SPL).

In expressing the volume of sound, in case the sound pressure isexpressed using the decibel (dB), the volume of sound may be expressedas dB sound pressure level (dB SPL).

The decibel (dB) may, for example, be 10*log(P1/P0), and log may refer,for example, to the common logarithm. The common logarithm may refer,for example, to the logarithm with base 10. P1 may refer, for example,to a measured value, and P0 may refer, for example, to a referencevalue. 10*log(P1/P0) may, for example, be a formula used in theviewpoint of power or energy. A formula used in the viewpoint of currentor voltage may, for example, be 20*log(P1/P0).

Although the decibel (dB) corresponds to a relative value, theelectronic apparatus 100 may generally set, as a reference point, soundpressure of 0.0002 dyn/cm² (=20 uPA, in which u may refer, for example,to 10⁻⁶) which is the lowest volume (0 dB SPL) that a person may hear,and 20 uPa may be used like an absolute value. Further, the electronicapparatus 100 may set, as a reference value (0 dB), 10⁽⁻¹²⁾W*m⁻² in theviewpoint of sound intensity, and 10⁽⁻¹²⁾W*m⁻² may be used like anabsolute value.

The electronic apparatus 100 may measure the measured value P1 as SIL orSPL based, for example, and without limitation, on an applicationprocessor (AP), an integrated circuit (IC), a digital-to-analogconverter (DAC), a codec chip, or the like, which may include, forexample, a processor related to a microphone, and directly convert SILor SPL into dB. Intensity and pressure may have a difference in a unitof absolute value. However, in the electronic apparatus 100 according tothe disclosure, intensity and pressure may have a relative concept in aunit of dB, and thus whether SIL or SPL is used may not affect theoperation significantly.

The first sound data and second sound data received through the firstmicrophone 121 and the second microphone 122, respectively, may bechanged by a weighted value or gain. Under the assumption that power ofa sound source does not change, sound intensity may be in inverseproportion to the square of a distance between the sound source and themicrophone.

The electronic apparatus 100 according to an example embodiment maychange sound data based on the volume of sound data and object positioninformation to thereby match a balance between image data and sounddata. Therefore, to a person who watches a video (including image dataand changed sound data) generated by the electronic apparatus 100, theimage data and the sound data may feel natural.

Simple components of the electronic apparatus 100 are illustrated anddescribed hereinabove, but various components may be further included inthe electronic apparatus 100 in an example implementation. This will bedescribed in greater detail below with reference to FIG. 3.

FIG. 3 is a block diagram illustrating an example configuration of theelectronic apparatus of FIG. 2.

Referring to FIG. 3, the electronic apparatus 100 may include the camera110, the first microphone 121, the second microphone 122, the processor(e.g., including processing circuitry) 130, the display 140, a userinterface (e.g., including user interface circuitry) 150, aninput/output interface (e.g., including input/output circuitry) 160, acommunication interface (e.g., including communication circuitry) 170,and a memory 180.

An\ overlapping description of the same operations of the camera 110,the first microphone 121, the second microphone 122, and the processor130 as those described above may not be repeated here.

The display 140 may be implemented by various types of displays such as,for example, and without limitation, a liquid crystal display (LCD), anorganic light emitting diode (OLED) display, a plasma display panel(PDP), or the like. A driving circuit, a backlight unit, and the like,that may be implemented in a form such as a-si thin film transistor(TFT), a low temperature poly silicon (LTPS) TFT, or an organic TFT(OTFT) may be included in the display 140. The display 140 may beimplemented by a touch screen combined with a touch sensor, a flexibledisplay, a three-dimensional (3D) display, or the like.

In addition, the display 140 according to an embodiment of thedisclosure may include a display panel that outputs an image, and abezel that houses the display panel. For example, the bezel according toan embodiment of the disclosure may include a touch sensor (notillustrated) for detecting a user interaction.

The user interface 150 may include various user interface circuitry, andmay, for example, be implemented by a device such as a button, a touchpad, a mouse, or a keyboard, or may be implemented by a touch screenthat may perform the above-described display function and an operationinput function. The button may be various types of buttons such as amechanical button, a touch pad, and a wheel formed in any region such asa front surface portion, a side surface portion, or a rear surfaceportion of a body appearance of the electronic apparatus 100.

The input/output interface 160 may include various input/outputcircuitry, and may include, for example, one or more of high definitionmultimedia interface (HDMI), mobile high-definition link (MHL),universal serial bus (USB), DisplayPort (DP), a thunderbolt, a videographics array (VGA) port, an RGB port, a D-subminiature (D-SUB), or adigital visual interface (DVI).

The input/output interface 160 may input and output at least one of anaudio signal or a video signal.

In some examples, the input/output interface 160 may include, asseparate ports, a port for inputting and outputting only an audio signaland a port for inputting and outputting only a video signal, or may beimplemented as one port for inputting and outputting both an audiosignal and a video signal.

The communication interface 170 may include various communicationcircuitry and may perform communication with various types of externaldevices in various types of communication manners. The communicationinterface 170 may include various communication modules, each includingvarious communication circuitry, such as, for example, and withoutlimitation, a Wi-Fi module, a Bluetooth module, an infraredcommunication module, a wireless communication module, and the like.Each communication module may, for example, be implemented in a form ofat least one hardware chip.

The Wi-Fi module and the Bluetooth module may perform communication in aWi-Fi manner and a Bluetooth manner, respectively. In case of using theWi-Fi module or Bluetooth module, various connection information such asa service set identifier (SSID) and a session key is first transmittedand received, communication is connected using the connectioninformation, and various information may then be transmitted andreceived.

The infrared communication module may perform communication according toan infrared data association (IrDA) technology using infrared lightwhich lies between visible light and millimeter waves for short-distancewireless data transmission.

The wireless communication module may include, for example, and withoutlimitation, at least one communication chip performing communicationaccording to various wireless communication protocols such as Zigbee,3^(rd) generation (3G), 3^(rd) generation partnership project (3GPP),long term evolution (LTE), LTE Advanced (LTE-A), 4^(th) generation (4G),and 5^(th) generation (5G), in addition to the communication mannerdescribed above.

In addition, the communication interface may include at least one wiredcommunication module performing communication using a local area network(LAN) module, an Ethernet module, a pair cable, a coaxial cable, anoptical fiber cable, an ultra wide-band (UWB) module, or the like.

According to an example, the communication interface 170 may use thesame communication module (for example, the Wi-Fi module) to performcommunication with an external device such as a remote controller, andan external server.

According to an example, the communication interface 170 may usedifferent communication modules (for example, the Wi-Fi module) toperform communication with an external device such as a remotecontroller, and an external server. For example, the communicationinterface 170 may use at least one of the Ethernet module or the Wi-Fimodule to perform communication with an external server, and may use theBT module to perform communication with an external device such as aremote controller. However, this is merely an example, and thecommunication interface 170 may use at least one of variouscommunication modules in case of performing communication with aplurality of external devices or external servers.

The memory 180 may be implemented by an internal memory such as, forexample, and without limitation, a read-only memory (ROM) (for example,an electrically erasable programmable read only memory (EEPROM)) orrandom access memory (RAM) included in the processor 130 or beimplemented by a memory separate from the processor 130. The memory 180may be implemented in a form of a memory embedded in the electronicapparatus 100 or in a form of a memory attachable to and detachable fromthe electronic apparatus 100, depending on a data storing purpose. Forexample, data for driving the electronic apparatus 100 may be stored inthe memory embedded in the electronic apparatus 100, and data for anextension function of the electronic apparatus 100 may be stored in thememory attachable to and detachable from the electronic apparatus 100.

The memory embedded in the electronic apparatus 100 may be implementedby, for example, and without limitation, at least one of a volatilememory (for example, a dynamic RAM (DRAM), a static RAM (SRAM), or asynchronous dynamic RAM (SDRAM)), a non-volatile memory (for example, aone time programmable ROM (OTPROM), a programmable ROM (PROM), anerasable and programmable ROM (EPROM), an EEPROM, a mask ROM, a flashROM, a flash memory (for example, a NAND flash or a NOR flash), a harddrive, or a solid state drive (SSD), and the memory attachable to anddetachable from the electronic apparatus 100 may be implemented by amemory card (for example, a compact flash (CF), a secure digital (SD), amicro secure digital (Micro-SD), a mini secure digital (Mini-SD), anextreme digital (xD), or a multi-media card (MMC)), an external memory(for example, a USB memory) connectable to a USB port, or the like.

FIG. 4 is a flowchart illustrating an example video recording operationaccording to an embodiment.

Referring to FIG. 4, the electronic apparatus 100 may capture an imageusing the camera 110. The electronic apparatus 100 may record a sound(or audio) using the first microphone 121 and the second microphone 122(S405). Respective sound data recorded through a plurality ofmicrophones may be used as stereo sound data.

The electronic apparatus 100 may identify a position where an objectcorresponding to a subject that generates a sound is displayed (S410).The object that makes (generates) a sound may refer, for example, to anobject (or subject) that makes a sound. For example, the object thatgenerates a sound may be a person who utters a voice, an animal thatmakes a sound, or a food that makes a boiling sound. The electronicapparatus 100 may obtain position information indicating the positionwhere the object that generates a sound is displayed based on image dataobtained through the camera 110. An operation of obtaining the positioninformation will be described in greater detail below with reference toFIG. 6.

The electronic apparatus 100 may change the recorded sound data based onthe obtained object position information (S415). The electronicapparatus 100 may record a sound using a plurality of microphones, andmay generate a plurality of sound data based on the sound recordedthrough each of the plurality of microphones. Further, the electronicapparatus 100 may change the plurality of generated sound data based onthe obtained object position information.

In FIG. 4, a case that the electronic apparatus 100 changes the recordedsound data has been illustrated. However, in an example implementation,the electronic apparatus 100 may obtain sound data by adjusting theinput gain of a microphone, rather than changing the recorded sounddata. For example, the input gain of a microphone is changed based onthe object position information, and the electronic apparatus 100 mayobtain sound data changed depending on the input gain of the microphone.

FIG. 5 is a flowchart illustrating an example recording operation ofFIG. 4 according to an embodiment.

Referring to FIG. 5, the electronic apparatus 100 may obtain image dataincluding an object (S505).

The electronic apparatus 100 may obtain first sound data and secondsound data (S510). The first sound data and the second sound data mayrefer, for example, to stereo sound data, and may be obtained intime-synchronization (time sync) with the image data. S505 and S510 maybe performed at the same time. For example, the electronic apparatus 100may obtain the image data, the first sound data, and the second sounddata at the same time.

The electronic apparatus 100 may obtain sound ratio information betweenthe volume of the first sound data and the volume of the second sounddata (S515). The sound ratio information may refer, for example, to aratio between the volumes of sounds recorded through a plurality ofmicrophones, respectively.

The electronic apparatus 100 may identify an object that generates asound (S520). The object that generates a sound may refer, for example,to an object that makes a sound. The electronic apparatus 100 mayidentify the object that generates a sound based on the image dataobtained through the camera 110. For example, the processor 100 mayidentify a plurality of objects in the obtained image data using anartificial intelligence module, and may identify an object that makes asound among the plurality of objects.

The electronic apparatus 100 may identify information on a positionwhere the identified object is displayed on the display (S525). Theobject identified by the electronic apparatus 100 is based on the imagedata, and the image data may be displayed on a device including adisplay. The image data may include position information correspondingto the object, and the position information may include informationindicating a position where the object is to be displayed in the image.The position information may refer, for example, to coordinateinformation.

The electronic apparatus 100 may obtain distance ratio information basedon the position information corresponding to the object (S530). Thedistance ratio information may refer, for example, to a ratio between adistance from the position where the object is displayed to a left axis,and a distance from the position where the object is displayed to aright axis. A description of the position information will be providedin greater detail below with reference to FIG. 6.

The electronic apparatus 100 may change volume information of the sounddata based on at least one of the sound ratio information obtained inS515 or the distance ratio information obtained in S530 (S535). Acalculation process of changing the sound data based on at least one ofthe sound ratio information or the distance ratio information will bedescribed in greater detail below with reference to FIG. 9.

FIG. 6 is a diagram illustrating an example operation of obtainingobject position information according to an embodiment.

A first example 600-1 in which an object 605 is positioned at the centerof image data and a second example 600-2 in which the object 605 ispositioned on the right side of the image data will be described withreference to FIG. 6.

The image data may have three reference axes. The reference axes mayinclude a left axis 610, an object axis 615, and a right axis 620. Theleft axis 610 may be a y axis (or vertical axis) corresponding to theleftmost side of the image data, and may refer, for example, to a y axiswith an x-coordinate value of 0 in the image data. The object axis 615may refer, for example, to a y axis with coordinates at which the objectis positioned. The right axis 620 may refer, for example, to a y axis(or vertical axis) corresponding to the rightmost side of the imagedata, and may refer, for example, to a y axis with a maximumx-coordinate value in the image data. In case the size of the image isuniform, the left axis 610 and the right axis 620 may be fixed, and theobject axis 615 may move according to movement of the object.

The left axis 610 and the right axis 620 may be obtained based ondisplay information of various display devices that output the imagedata, in addition to the image data. The object 605 may refer, forexample, to an object that generates a sound. Although FIG. 6illustrates a case that the object is a person, the object 605 may bevarious objects that make a sound.

Position information of the object 605 may refer, for example, tocoordinate information, and the electronic apparatus 100 may obtain thefirst distance information 601 and the second distance information 602based on the coordinate information of the object 605.

The first distance information 601 may include distance informationbetween the object axis 615 of the image data and the left axis 610 ofthe image data (DL). Further, the first distance information 601 mayrefer, for example, to an x-coordinate value of the object.

The second distance information 602 may include distance informationbetween the object axis 615 of the image data and the right axis 620 ofthe image data (DR). Further, the second distance information 602 mayrefer, for example, to a value obtained by subtracting the x-coordinatevalue of the object from a maximum x-coordinate value of the image data.

In the first example 600-1, the object 605 may be positioned at thecenter of the displayed image data. The first distance information 601and the second distance information 602 may be the same value. Forexample, in case a maximum x-axis value of the image data is 10, and thex-coordinate value of the object is 5, the first distance information601 may be 5, and the second distance information 602 may be 5.

In the second example 600-2, the object 605 may be positioned on theright side of the displayed image data. The first distance information601 may be greater than the second distance information 602. Forexample, in case the maximum x-axis value of the image data is 10, andthe x-coordinate value of the object is 8, the first distanceinformation 601 may be 8, and the second distance information 602 may be2.

The electronic apparatus 100 may obtain distance ratio information basedon the first distance information 601 and the second distanceinformation 602. The distance ratio information may include at least oneof a value obtained by dividing the first distance information 601 bythe second distance information 602, or a value obtained by dividing thesecond distance information 602 by the first distance information 601.

FIG. 7 is a diagram illustrating an example of a change of sound dataaccording to an embodiment.

Referring to FIG. 7, it may be assumed by way of non-limiting examplethat the first microphone 121 is a left-side microphone and the secondmicrophone 122 is a right-side microphone. Such left-right division isunder the assumption that the camera 110 and the first microphone 121are positioned on the left side and the second microphone 122 ispositioned on the right side as in the electronic apparatus 100illustrated in FIG. 1.

However, the camera 110 and the first microphone 121 may be positionedon the right side. A description thereof will be provided in greaterdetail below with reference to FIG. 18.

Referring to FIG. 7, the electronic apparatus 100 may generate stereosound data using first sound data obtained through the first microphone121, and second sound data obtained through the second microphone 122.

The electronic apparatus 100 may obtain first weighted value information712 corresponding to the first sound data and second weighted valueinformation 722 corresponding to the second sound data.

According to a first example 700-1, the electronic apparatus 100 mayobtain predetermined weighted value information.

The first weighted value information 712 may be a value obtained bymultiplying a predetermined constant (K) and a reciprocal (1/PL) of thevolume 711 of the first sound data. The second weighted valueinformation 722 may be a value obtained by multiplying the predeterminedconstant (K) and a reciprocal (1/PR) of the volume 721 of the secondsound data. The predetermined constant (K) may be a value predeterminedto prevent and/or reduce the volume of the sound from being decreased orincreased at the time of changing the volume of the sound, or to providean optimal volume of sound. According to a second example 700-2, theelectronic apparatus 100 may obtain weighted value information based ondistance information.

The first weighted value information 712 may be a value obtained bymultiplying a value (K1) determined based on first distance information,and the reciprocal (1/PL) of the volume 711 of the first sound data.

The second weighted value information 722 may be a value obtained bymultiplying a value (K2) determined based on second distanceinformation, and the reciprocal (1/PR) of the volume 721 of the secondsound data.

The electronic apparatus 100 may obtain the volume (PL) 711 of the firstsound data obtained through the first microphone 121. Here, theelectronic apparatus 100 may obtain the first weighted value information(k1/PL) 712 corresponding to the first sound data. “K1” may be a valuedetermined based on object position information. The electronicapparatus 100 may change the first sound data by multiplying the volume711 of the first sound data by the first weighted value information 712.The changed first sound data may include “K1”. For example, theelectronic apparatus 100 may obtain the volume (K1) 713 of the firstsound data changed based on the volume 711 of the first sound data andthe first weighted value information 712 corresponding to the firstsound data.

The electronic apparatus 100 may obtain the volume (PR) 721 of thesecond sound data obtained through the second microphone 122. Here, theelectronic apparatus 100 may obtain the second weighted valueinformation (K2/PR) 722 corresponding to the second sound data. “K2” maybe a value determined based on object position information. Here, theelectronic apparatus 100 may change the second sound data by multiplyingthe volume 721 of the second sound data by the second weighted valueinformation 722. The changed second sound data may include “K2”. Forexample, the electronic apparatus 100 may obtain the volume (K2) 723 ofthe second sound data changed based on the volume 721 of the secondsound data and the second weighted value information 722 correspondingto the second sound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 731, and obtain “K1:K2” as post-change sound ratioinformation 733.

FIG. 8 is a diagram illustrating an example of a change of sound dataaccording to an embodiment.

Referring to FIG. 8, it may be assumed by way of non-limiting examplethat the first microphone 121 is a left-side microphone and the secondmicrophone 122 is a right-side microphone. Other overlappingdescriptions provided with reference to FIG. 7 may not be repeated here.

The electronic apparatus 100 may obtain first weighted value information812 corresponding to the first sound data and second weighted valueinformation 822 corresponding to the second sound data.

The first weighted value information 812 may be a value obtained bymultiplying a reciprocal (1/DL) of first distance information DL, and areciprocal (1/PL) of the volume 811 of the first sound data. “K1” ofFIG. 7 may refer, for example, to “1/DL”.

The second weighted value information 822 may be a value obtained bymultiplying a reciprocal (1/DR) of second distance information DR, and areciprocal (1/PR) of the volume 821 of the second sound data. “K2” ofFIG. 7 may refer, for example, to “1/DR”.

The electronic apparatus 100 may obtain the volume (PL) 811 of the firstsound data obtained through the first microphone 121. The electronicapparatus 100 may obtain the first weighted value information(1/(PL*DL)) 812 corresponding to the first sound data. “1/(PL*DL)” maybe a value determined based on object position information. Theelectronic apparatus 100 may change the first sound data by multiplyingthe volume 811 of the first sound data by the first weighted valueinformation 812. The changed first sound data may include “1/(PL*DL)”.For example, the electronic apparatus 100 may obtain the volume (1/DL)813 of the first sound data changed based on the volume 811 of the firstsound data and the first weighted value information 812 corresponding tothe first sound data.

The electronic apparatus 100 may obtain the volume (PR) 821 of thesecond sound data obtained through the second microphone 122. Theelectronic apparatus 100 may obtain the second weighted valueinformation (1/(PR*DR)) 822 corresponding to the second sound data.“1/(PR*DR)” may be a value determined based on object positioninformation. The electronic apparatus 100 may change the second sounddata by multiplying the volume 821 of the second sound data by thesecond weighted value information 822. The changed second sound data mayinclude “1/(PR*DR)”. For example, the electronic apparatus 100 mayobtain the volume (1/DR) 823 of the second sound data changed based onthe volume 821 of the second sound data and the second weighted valueinformation 822 corresponding to the second sound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 831, and obtain “DR:DL” as post-change sound ratioinformation 833.

Although FIGS. 7 and 8 illustrate the operation of changing both thevolume of the first sound data and the volume of the second sound data,only one of the volume of the first sound data or the volume of thesecond sound data may be changed in some implementation examples. Adescription thereof will be provided in greater detail below withreference to FIG. 9.

FIG. 9 is a diagram illustrating an example of a change of sound dataaccording to an embodiment.

A first example 900-1 in which only the volume 911 of the first sounddata is changed, and a second example 900-2 in which only the volume 921of the second sound data is changed will be described with reference toFIG. 9.

The electronic apparatus 100 may obtain first weighted value information912 corresponding to the first sound data and second weighted valueinformation 922 corresponding to the second sound data.

The first weighted value information 912 may be a value obtained bymultiplying first distance ratio information (DR/DL) corresponding tothe first sound data, and first sound ratio information (PR/PL)corresponding to the first sound data. In case the first distance ratioinformation is DL/DR, the first sound ratio information corresponding tothe first sound data may be PL/PR.

The second weighted value information 922 may be a value obtained bymultiplying second distance ratio information (DL/DR) corresponding tothe second sound data, and second sound ratio information (PL/PR)corresponding to the second sound data. In case the second distanceratio information is DR/DL, the second sound ratio informationcorresponding to the second sound data may be PR/PL.

According to the first example 900-1, the electronic apparatus 100 maychange only the volume 911 of the first sound data without changing thevolume 921 of the second sound data. Therefore, the volume (PR) 921 ofthe second sound data is maintained as same value (PR) 923.

The electronic apparatus 100 may obtain the volume (PL) 911 of the firstsound data obtained through the first microphone 121. Here, theelectronic apparatus 100 may obtain the first weighted value information((DR/DL)*(PR/PL)) 912 corresponding to the first sound data.“(DR/DL)*(PR/PL)” may be a value obtained based on the distance ratioinformation corresponding to the first sound data and the sound ratioinformation corresponding to the first sound data. The electronicapparatus 100 may change the first sound data by multiplying the volume911 of the first sound data by the first weighted value information 912.The changed first sound data may include “(DR/DL)*PR”. For example, theelectronic apparatus 100 may obtain the volume ((DR/DL)*PR) 913 of thefirst sound data changed based on the volume 911 of the first sound dataand the first weighted value information 912 corresponding to the firstsound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 931, and obtain “DR:DL” as post-change sound ratioinformation 933.

According to the second example 900-2, the electronic apparatus 100 maychange only the volume 921 of the second sound data without changing thevolume 911 of the first sound data. Therefore, the volume (PR) 911 ofthe first sound data is maintained as same value (PR) 913.

The electronic apparatus 100 may obtain the volume (PR) 921 of thesecond sound data obtained through the second microphone 122. Here, theelectronic apparatus 100 may obtain the second weighted valueinformation ((DL/DR)*(PL/PR)) 922 corresponding to the second sounddata. Here, “(DL/DR)*(PL/PR)” may be a value obtained based on thedistance ratio information corresponding to the second sound data andthe sound ratio information corresponding to the second sound data. Theelectronic apparatus 100 may change the second sound data by multiplyingthe volume 921 of the second sound data by the second weighted valueinformation 922. The changed second sound data may include “(DL/DR)*PL”.That is, the electronic apparatus 100 may obtain the volume ((DL/DR)*PL)923 of the second sound data changed based on the volume 921 of thesecond sound data and the second weighted value information 922corresponding to the second sound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 931, and obtain “DR:DL” as post-change sound ratioinformation 933.

Meanwhile, FIGS. 7, 8 and 9 illustrate various examples in whichobtained volume information of sound data is multiplied by weightedvalue information. However, in some examples, the electronic apparatus100 may change volume information of sound data using various methods.

As an example, the electronic apparatus 100 may subtract a first value(a dB) from the volume of the first sound data and subtract a secondvalue (b dB) from the volume of the second sound data based on apredefined decibel formula. The electronic apparatus 100 may amplify therespective obtained values by a third value (k dB).

As another example, in the electronic apparatus 100, different weightedvalue information, correction algorithm (sound volume information changealgorithm), or the like may be applied depending on a difference of astereo microphone element, a processing method of a terminal device, orthe like. For example, a reciprocal, the square of a reciprocal, adistance square ratio, or the like may be applied to the correctionalgorithm.

The algorithm of changing volume information of sound data in thecontrol operation of the electronic apparatus 100 according to thedisclosure is not limited to any specific algorithm, and variousalgorithms may be applied according to user setting or situation.

FIG. 10 is a flowchart illustrating an example operation of theelectronic apparatus illustrated in FIG. 9 according to an embodiment.

Referring to FIG. 10, the electronic apparatus 100 may obtain image data(S1005). The electronic apparatus 100 may obtain first sound data andsecond sound data (S1010). The first sound data and the second sounddata may refer, for example, to stereo sound data. For example, thefirst sound data may refer, for example, to left stereo sound data, andthe second sound data may refer, for example, to right stereo sounddata.

The electronic apparatus 100 may determine whether the volume of thefirst sound data is higher than the volume of the second sound data(S1015). A reason for obtaining the volume of sound data may, forexample, be to change only one of the first sound data or the secondsound data. In general, sound quality may deteriorate as the volume ofsound data is increased. However, the deterioration in sound quality maybe less severe in case of decreasing the volume of sound data.Therefore, the electronic apparatus 100 may perform an operation ofdecreasing the volume of sound data having a higher volume among thefirst sound data and the second sound data.

In case the volume of the first sound data is higher than the volume ofthe second sound data (“Y” in operation S1015), the electronic apparatus100 may obtain a first weighted value corresponding to the first sounddata (S1020). The electronic apparatus 100 may change the volume of thefirst sound data by multiplying the volume of the first sound data bythe first weighted value (S1025).

In case the volume of the first sound data is lower than the volume ofthe second sound data (“N” in operation S1015), the electronic apparatus100 may obtain a second weighted value corresponding to the second sounddata (S1030). The electronic apparatus 100 may change the volume of thesecond sound data by multiplying the volume of the second sound data bythe second weighted value (S1035).

FIG. 11 is a diagram illustrating an example recording operationaccording to an embodiment.

Referring to FIG. 11, the electronic apparatus 100 may record a video.For example, the electronic apparatus 100 may capture an image using thecamera 110. The camera 110 may refer, for example, to a front camera,and it is assumed by way of non-limiting example that a speaker 1105stands in front of the camera 110. The speaker 1105 stands in front ofthe camera 110, but does not stand in front of the center of theelectronic apparatus 100.

Further, in case the speaker 1105 generates a sound, the firstmicrophone 121 and the second microphone 122 may record the soundgenerated by the speaker 1105. A distance between the speaker 1105 andthe first microphone 121 may be different from that between the speaker1105 and the second microphone 122. Therefore, the volume of first sounddata recorded through the first microphone 121 and the volume of secondsound data recorded through the second microphone 122 may be differentfrom each other.

Although an object 1110 corresponding to the speaker 1105 is positionedat the center of the image, the volume of the first sound data and thevolume of the second sound data may be different from each other. Insuch a situation, a person who watches the video may misunderstand thatthe sound data has a problem. Therefore, the electronic apparatus 100may perform an operation of changing the volume of sound data to solvesuch an imbalance problem.

For example, it is assumed by way of non-limiting example that a ratiobetween the volume of the first sound data and the volume of the secondsound data is 2:0.5. Because the object 1110 corresponding to thespeaker 1105 is positioned at the center of the image data, theelectronic apparatus 100 may change the sound data to make the volume ofthe first sound data and the volume of the second sound data have aratio of 1:1. The specific sound data conversion (change) process hasbeen described with reference to FIGS. 7, 8 and 9, and thus anoverlapping description may not be repeated here.

The electronic apparatus 100 may generate information on pre-conversionsound data and information on post-conversion sound data (orpost-changed sound data), and include, in the image data, the generatedinformation. The information on sound data may refer, for example, tovolume information or volume ratio information. The electronic apparatus100 may display, on the display, at least one of the pre-conversionsound data volume ratio (2:0.5) or the post-conversion sound data volumeratio (1:1).

The electronic apparatus 100 may display volume information as it iswith a value of the volume of sound data, a dB value, or the like (notillustrated as a UI).

The electronic apparatus 100 may display, as a UI 1115, the informationon pre-conversion sound data and the information on post-conversionsound data. “Obtained” may refer, for example, to an actual volume orvolume ratio of an obtained sound (raw signal), and “changed” may refer,for example, to a corrected volume or volume ratio of a changed andrecorded sound. Further, the recorded sound may be obtained by recordingand changing a sound based on the algorithm according to the disclosure,and may be obtained by adjusting the gain at the time of obtaining asound and recording data based on the adjustment.

FIG. 12 is a diagram illustrating an example recording operationaccording to an embodiment.

Referring to FIG. 12, the electronic apparatus 100 may record a video ina situation in which a speaker 1205 stands in front of the center of afront surface portion of the electronic apparatus 100.

In case the speaker 1205 stands in front of the center of the frontsurface portion of the electronic apparatus 100, an object 1210corresponding to the speaker 1205 may not necessarily be positioned atthe center of image data captured by the camera 110. This is because thespeaker 1205 is on the left of the camera 110. In case the left-rightreversal function is automatically performed in the camera 110, theimage data in which the object 1210 corresponding to the speaker 1205 ispositioned on the right side may be generated.

The volume of sound data recorded through the first microphone 121 andthe volume of sound data recorded through the second microphone 122 maybe the same as each other (under the assumption that the firstmicrophone 121 and the second microphone 122 have the same performance).This is because a distance between the speaker 1205 and the firstmicrophone 121 is the same as that between the speaker 1205 and thesecond microphone 122. In case the first microphone 121 and the secondmicrophone 122 have different performances, a compensation based on theperformance difference may be added.

Although the object 1210 corresponding to the speaker 1205 is positionedon the right side, a person who watches the video may feel that thevolume of first sound data and the volume of second sound data are thesame as each other. Therefore, the person who watches the video may feelthat the stereo sound data and the image data do not match each other.

Therefore, the electronic apparatus 100 may change sound data. Forexample, the electronic apparatus 100 may perform an operation ofchanging the volume of sound data based on position information of theobject 1210 corresponding to the speaker 1205.

For example, it is assumed by way of non-limiting example that a ratiobetween the volume of the first sound data and the volume of the secondsound data is 1:1. Because the object 1210 corresponding to the speaker1205 is positioned at a position corresponding to a ratio of 2(left):0.5 (right) in the image data, the electronic apparatus 100 maychange the sound data to make the volume of the first sound data and thevolume of the second sound data have a ratio of 0.5:2. The specificsound data conversion (change) process has been described with referenceto FIGS. 7, 8 and 9, and thus an overlapping description may not berepeated here.

The electronic apparatus 100 may display, on the display, at least oneof the pre-conversion sound data volume ratio (1:1) or thepost-conversion sound data volume ratio (0.5:2). The electronicapparatus 100 may display, as a UI 1215, the information onpre-conversion sound data and the information on post-conversion sounddata. The UI has been described with reference to FIG. 11, and thus anoverlapping description may not be repeated here.

FIG. 13 is a diagram illustrating an example recording operationaccording to an embodiment.

Referring to FIG. 13, it is assumed by way of non-limiting example thata speaker 1305 stands in front of a rear camera (not illustrated).Unlike FIG. 11 in which it is assumed by way of non-limiting examplethat the speaker 1105 stands in front of the front camera 110 (or selfiecamera), a situation in which the speaker 1305 stands in front of therear camera (not illustrated) is assumed by way of non-limiting examplein FIG. 13.

The speaker 1305 does not stand in front of the electronic apparatus100, volume information of first sound data and volume information ofsecond sound data may be different from each other. Therefore, theelectronic apparatus 100 may perform an operation of changing at leastone of the volume of the first sound data or the volume of the secondsound data.

For example, it is assumed for convenience of illustration that a ratiobetween the volume of the first sound data and the volume of the secondsound data is 2:0.5. Because an object 1310 corresponding to the speaker1305 is positioned at the center of image data, the electronic apparatus100 may change the sound data to make the volume of the first sound dataand the volume of the second sound data have a ratio of 1:1. Thespecific sound data conversion (change) process has been described withreference to FIGS. 7, 8 and 9, and thus an overlapping description maynot be repeated here.

The electronic apparatus 100 may display, on the display, at least oneof the pre-conversion sound data volume ratio (2:0.5) or thepost-conversion sound data volume ratio (1:1). The electronic apparatus100 may display, as a UI 1315, the information on pre-conversion sounddata and the information on post-conversion sound data.

FIG. 14 is a diagram illustrating an example recording operationaccording to an embodiment of the disclosure.

Referring to FIG. 14, the electronic apparatus 100 may record a video ina situation in which a speaker 1405 stands in front of the center of arear surface portion of the electronic apparatus 100.

The electronic apparatus 100 may obtain image data using the rear camera(not illustrated), and the image data in which an object 1410corresponding to the speaker 1405 is displayed on the right side of theentire image may be generated. The volume of first sound data and thevolume of second sound data may be the same as each other.

The electronic apparatus 100 may perform an operation of changing atleast one of the volume of the first sound data or the volume of thesecond sound data.

For example, it is assumed for convenience of illustration that a ratiobetween the volume of the first sound data and the volume of the secondsound data is 1:1. Because the object 1410 corresponding to the speaker1405 is positioned at a position corresponding to a ratio of 2(left):0.5 (right) in the image data, the electronic apparatus 100 maychange the sound data to make the volume of the first sound data and thevolume of the second sound data have a ratio of 0.5:2. The specificsound data conversion (change) process has been described with referenceto FIGS. 7, 8 and 9, and thus an overlapping description may not berepeated here.

The electronic apparatus 100 may display, on the display, at least oneof the pre-conversion sound data volume ratio (1:1) or thepost-conversion sound data volume ratio (0.5:2). The electronicapparatus 100 may display, as a UI 1415, the information onpre-conversion sound data and the information on post-conversion sounddata.

FIG. 15 is a diagram illustrating an example UI used for a controloperation of the electronic apparatus according to an embodiment.

Referring to FIG. 15, the electronic apparatus 100 may display UIs 1505and 1510 that guide a speaker to be positioned at the center of a cameraand make a sound.

The electronic apparatus 100 may display a real-time image captured by acamera. Further, the electronic apparatus 100 may guide a speaker to bepositioned at the center of the real-time image and make a sound.Positioning the speaker at the center of the real-time image may refer,for example to the speaker standing in front of the camera. Further, theelectronic apparatus 100 may guide the speaker to make a sound. Thereason to guide the speaker to make sound is to compare sound datarecorded through the first microphone 121 and the second microphone 122,respectively. In general, the camera is positioned close to only one ofthe first microphone 121 or the second microphone 122, and thus theelectronic apparatus 100 may guide the speaker to make a sound in frontof the center of the camera to analyze the difference.

For such a guide, the electronic apparatus 100 may provide the UI 1505that displays text information, and the image UI 1510 that guides thespeaker to be positioned at the center of the real-time image.

A specific example operation therefor will be described in greaterdetail below with reference to FIG. 27.

The UI 1510 of FIG. 15 may be used to obtain a reference (referencepoint) for volume information of sound data actually received throughthe first microphone 121 and the second microphone 122 in a situation inwhich the sound data should be output at a left-right ratio of 1:1 (forexample, FIG. 11). The UI 1510 may be used to obtain the reference atthe beginning of recording a video by the user. The UI 1510 may be asetting item of options of video recording.

FIG. 16 is a diagram illustrating an example UI that may be used for anoperation of converting sound data according to an embodiment.

Referring to FIG. 16, the electronic apparatus 100 may provide a UI 1605for manually changing a volume ratio of sound data. The electronicapparatus 100 may provide a UI 1610 for controlling adjustment of thevolume ratio. In addition, the electronic apparatus 100 may provide a UI1615 that provides information on pre-change sound data and informationon post-change sound data.

In a first example 1600-1, a second example 1600-2, and a third example1600-3, it is assumed by way of non-limiting example that an objectcorresponding to a speaker is positioned on the right side of imagedata, and first sound data and second sound data are obtained at avolume ratio of 1:1.

The operation of changing sound data may be automatically performed.However, in FIG. 16, an embodiment in which the automatic changeoperation is performed first, and an operation of manually changingsound data is additionally performed is described.

The electronic apparatus 100 may receive a user input. A user may changea ratio of sound data through the UI 1610.

According to the first example 1600-1, the electronic apparatus 100 mayprovide changed ratio information of sound data based, for example, onposition information of an object corresponding to a speaker, and theelectronic apparatus 100 may perform a control to make the UI 1610 bepositioned based on the provided changed ratio information. In case theposition information of the object corresponding to the speakerindicates that a left-right distance information ratio is 2:0.5, theelectronic apparatus 100 may determine post-change sound ratioinformation as 0.5:2. The electronic apparatus 100 may provide thepost-change sound ratio information (0.5:2) through the UI 1615.Further, the electronic apparatus 100 may determine a position where theUI 1610 is displayed to correspond to the post-change sound ratioinformation (0.5:2) in the UI 1605. For example, the electronicapparatus 100 may perform a control to make the UI 1610 correspond tothe post-change sound ratio information (0.5:2) in the UI 1605.

The electronic apparatus 100 may additionally display UIs 1620, 1625,1630, and 1635 for changing the volume of sound data. The electronicapparatus 100 may additionally display the UI 1620 for increasing thevolume of the first sound data, the UI 1625 for decreasing the volume ofthe first sound data, the UI 1630 for increasing the volume of thesecond sound data, and the UI 1635 for decreasing the volume of thesecond sound data.

In case an input indicating that the UI 1620 is touched is received, theelectronic apparatus 100 may increase the volume of the first sounddata. Further, the electronic apparatus 100 may change the position ofthe UI 1610 based on the increased volume of the first sound data anddisplay the UI 1610 at the changed position. According to the firstembodiment 1600-1, in case an input indicating that the UI 1620 istouched is received, the electronic apparatus 100 may move the UI 1610to the left.

According to the second example 1600-2, the electronic apparatus 100 mayreceive a user input of adjusting the UI 1610. It is assumed by way ofnon-limiting example that the electronic apparatus 100 receives a userinput of adjusting the UI 1610 to be positioned at the center of the UI1605. The electronic apparatus 100 may analyze the user input. Forexample, the electronic apparatus 100 may compare left-side distanceinformation and right-side distance information of the position of theUI 1610 in the UI 1605. According to the second example 1600-2, distanceratio information of the left-side distance information and theright-side distance information of the UI 1610 may be 1:1. Theelectronic apparatus 100 may determine the post-change sound ratioinformation based on the distance ratio information of the UI 1610. Forexample, the electronic apparatus 100 may determine the post-changesound ratio information as 1:1 to correspond to the distance ratioinformation (1:1) of the UI 1610, and may change the sound data based onthe determined post-change sound ratio information (1:1). The electronicapparatus 100 may provide the post-change sound ratio information (1:1)through the UI 1615. The UIs 1620 to 1635 have been described in thefirst example 1600-1, and thus an overlapping description thereof maynot be repeated here.

According to the third example 1600-3, it is assumed by way ofnon-limiting example that the electronic apparatus 100 receives a userinput of adjusting the UI 1610 to be positioned on the left side of theUI 1605. According to the third example 1600-3, distance ratioinformation of the left-side distance information and the right-sidedistance information of the UI 1610 may be 2:0.5. The electronicapparatus 100 may determine the post-change sound ratio information as2:0.5 to correspond to the distance ratio information (2.0.5) of the UI1610, and may change the sound data based on the determined post-changesound ratio information (2.0.5). The electronic apparatus 100 mayprovide the post-change sound ratio information (2.0.5) through the UI1615. The UIs 1620 to 1635 have been described in the first example1600-1, and thus an overlapping description thereof may not be repeatedhere.

Note that various UIs illustrated in FIG. 16 may be simultaneouslydisplayed, or any one of the UIs may be displayed. For example, theelectronic apparatus 100 may display at least one of the UIs 1605 and1610 for changing the volume of sound data or the UIs 1620 to 1635 forchanging the volume of sound data.

FIG. 17 is a diagram illustrating an example operation of convertingsound data using a left-right reversal mode according to an embodiment.

A first example 1700-1 in which the left-right reversal mode isactivated and a second example 1700-2 in which the left-right reversalmode is not activated will be described with reference to FIG. 17.

The left-right reversal mode may include a function of reversing theleft and right of an image in case the front camera captures the image.

In the first example 1700-1, the electronic apparatus 100 may display areal-time captured image on the display. The electronic device 100 mayprovide a left-right reversed image. An image of a speaker 1705 may becaptured on the left of the camera 110. In case the left-right reversalmode is activated, the electronic apparatus 100 may display image datain which an object 1710 corresponding to the speaker 1705 is positionedon the right side of the image. Further, the electronic apparatus 100may change sound data based on a position of the object 1710. Theelectronic apparatus 100 may provide post-change sound ratio informationthrough an UI 1715.

In the second example 1700-2, the electronic apparatus 100 may provideimage data without activating the left-right reversal mode. For example,in case the left-right reversal mode is not activated, the electronicapparatus 100 may display image data in which the object 1710corresponding to the speaker 1705 is positioned on the left side of theimage. Further, the electronic apparatus 100 may change sound data basedon a position of the object 1710. The electronic apparatus 100 mayprovide post-change sound ratio information through the UI 1715.

The electronic apparatus 100 may provide different sound ratioinformation based on whether the left-right reversal mode is activated.Further, a different sound volume change algorithm may be reflectedbased on whether the left-right reversal mode is activated.

In the left-right reversal mode according to the disclosure, in case animage of the speaker 1705 is captured on the left of the camera, theimage of the speaker 1705 is displayed on the right side of the imagedata. For example, with the left-right reversal function, the object1710 corresponding to the speaker 1705 may be displayed on the left sideor right side of the image data. The present disclosure describes thatthe object 1710 corresponding to the speaker 1705 is displayed in aright region in case the left-right reversal mode is ON. However, insome examples, the object 1710 corresponding to the speaker 1705 may bedisplayed in a left region even in case the left-right reversal mode isON. This is because left and right may be reversed depending on whetherthe left-right reversal is made with respect to the camera or the user.

FIG. 18 is a diagram illustrating an example operation of convertingsound data according to a position of a camera according to anembodiment.

Referring to a first diagram 1800-1 of FIG. 18, a position of the camera110 included in the electronic apparatus 100 may be different. Thelengths of the electronic apparatus 100 in a vertical direction and ahorizontal direction may be different, and in case the electronicapparatus 100 is in a horizontal position, a position where the camerais disposed may be different. The camera 110 may be positioned on theleft side or the right side under the assumption that the electronicapparatus 100 is in a horizontal position. This is because theelectronic apparatus 100 may be upside down. For example, FIG. 11illustrates the embodiment in which video recording is performed in astate in which the electronic apparatus 100 is in a horizontal position,and the camera 110 is positioned on the left side. On the other hand,FIG. 18 illustrates an embodiment in which video recording is performedin a state in which the electronic apparatus 100 is in a horizontalposition, and the camera 110 is positioned on the right side.

FIG. 11 and FIG. 18 are different in regard to a method of using sounddata. For example, in FIG. 11, the first sound data obtained through thefirst microphone 121 is used as the left stereo sound data, and thesecond sound data obtained through the second microphone 122 is used asthe right stereo sound data. However, in FIG. 18, first sound dataobtained through the first microphone 121 is used as the right stereosound data, and second sound data obtained through the second microphone122 is used as the left stereo sound data.

This is because left and right positions of the microphones 121 and 122of FIG. 18 are changed, unlike the microphones 121 and 122 of FIG. 11.The electronic apparatus 100 may consider such a difference in usingsound data included in a video.

For example, the first microphone 121 may be a right-side microphone andthe second microphone 122 may be a left-side microphone. This may bereflected in the sound data change operation.

Referring to a second diagram 1800-2 of FIG. 18, the electronicapparatus 100 may obtain the volume (PL) 1811 of the second sound dataobtained through the second microphone 122. The electronic apparatus 100may obtain second weighted value information (1/(PL*DL)) 1812corresponding to the second sound data. Here, “1/(PL*DL)” may be a valuedetermined based on object position information. The electronicapparatus 100 may change the second sound data by multiplying the volume1811 of the second sound data by the second weighted value information1812. The changed second sound data may include “1/(PL*DL)”. Forexample, the electronic apparatus 100 may obtain the volume (1/DL) 1813of the second sound data changed based on the volume 1811 of the secondsound data and the second weighted value information 1812 correspondingto the second sound data. Further, the electronic apparatus 100 maygenerate the left stereo sound data using the volume (1/DL) 1813 of thechanged second sound data.

The electronic apparatus 100 may obtain the volume (PR) 1821 of thefirst sound data obtained through the first microphone 121. Here, theelectronic apparatus 100 may obtain first weighted value information(1/(PR*DR)) 1822 corresponding to the first sound data. “1/(PR*DR)” maybe a value determined based on object position information. Theelectronic apparatus 100 may change the first sound data by multiplyingthe volume 1821 of the first sound data by the first weighted valueinformation 1822. The changed first sound data may include “1/(PR*DR)”.For example, the electronic apparatus 100 may obtain the volume (1/DR)1823 of the first sound data changed based on the volume 1821 of thefirst sound data and the first weighted value information 1822corresponding to the first sound data. Further, the electronic apparatus100 may generate the right stereo sound data using the volume (1/DR)1823 of the changed first sound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 1831, and obtain “DR:DL” as post-change sound ratioinformation 1833.

FIG. 19 is a diagram illustrating an example operation of convertingsound data based on distance information between the electronicapparatus and an object according to an embodiment.

Referring to FIG. 19, the electronic apparatus 100 may provide changesound data in consideration of a distance between a speaker 1905 and theelectronic apparatus 100. For example, the electronic apparatus 100 mayidentify distance (d) information 1920 between the speaker 1905 and theelectronic apparatus 100 based on an object 1910 captured through thecamera. As an example, the electronic apparatus 100 may identify thedistance information 1920 based on a reference image. As illustrated inFIG. 15, the electronic apparatus 100 may obtain the reference image atthe beginning of recording a video, and may determine that the distancebetween the speaker 1905 and the electronic apparatus 100 is increasedin case the object 1910 becomes less than the reference image. Asanother example, the electronic apparatus 100 may obtain the distanceinformation 1920 using a time of flight (TOF) sensor module. As stillanother example, the electronic apparatus 100 may obtain the distanceinformation 1920 using multiple camera lenses. However, the disclosureis not necessarily limited to the above-described examples, and thedistance information 1920 may be obtained using various existingtechnologies.

The electronic apparatus 100 may change sound data based on theidentified distance information 1920. The electronic apparatus 100 mayprovide post-change sound ratio information through an UI 1915.

According to an embodiment, the electronic apparatus 100 may perform acontrol to decrease the volume of sound data as the distance information1920 between the speaker 1905 and the electronic apparatus 100 isincreased. A calculation process will be described in greater detailbelow with reference to FIG. 20.

According to an embodiment, the electronic apparatus 100 does not haveto perform the sound data change operation in case the distance betweenthe speaker 1905 and the electronic apparatus 100 is equal to or greaterthan a threshold distance. In case the distance information 1920 isequal to or greater than the threshold distance, there may be nosubstantial difference in balance between image data and sound data.Since the sound data change operation is not performed in the entirevideo recording process, the electronic apparatus 100 may increase amemory processing speed and efficiently manage power.

FIG. 20 is a diagram illustrating example sound data change operation ofFIG. 19 according to an embodiment.

Referring to FIG. 20, the electronic apparatus 100 may obtain the volume(PL) 2011 of the first sound data obtained through the first microphone121. The electronic apparatus 100 may obtain first weighted valueinformation (1/(PL*DL*d)) 2012 corresponding to first sound data.“1/(PL*DL*d)” may be a value determined based on object positioninformation. The electronic apparatus 100 may change the first sounddata by multiplying the volume 2011 of the first sound data by the firstweighted value information 2012. The changed first sound data mayinclude “1/(PL*DL*d)”. For example, the electronic apparatus 100 mayobtain the volume (1/DL*d) 2013 of the first sound data changed based onthe volume 2011 of the first sound data and the first weighted valueinformation 2012 corresponding to the first sound data.

The electronic apparatus 100 may obtain the volume (PR) 2021 of secondsound data obtained through the second microphone 122. The electronicapparatus 100 may obtain second weighted value information (1/(PR*DR*d))2022 corresponding to the second sound data. “1/(PR*DR*d)” may be avalue determined based on object position information. The electronicapparatus 100 may change the second sound data by multiplying the volume2021 of the second sound data by the second weighted value information2022. The changed second sound data may include “1/(PR*DR*d)”. Forexample, the electronic apparatus 100 may obtain the volume (1/DR*d)2023 of the second sound data changed based on the volume 2021 of thesecond sound data and the second weighted value information 2022corresponding to the second sound data.

The electronic apparatus 100 may obtain “PL:PR” as initial sound ratioinformation 2031, and obtain “DR:DL” as post-change sound ratioinformation 2033.

Although FIG. 20 illustrates an example in which 1/d is applied toweighted value information, 1/(d)² may be applied in some implementationexamples. A value (1/d or 1/(d)²) to be applied may be defined based ona predetermined user setting.

The distance information 1920 may be reflected in the first example700-1 of FIG. 7. The electronic apparatus 100 may change volumeinformation of sound data using k/d (or k/(d)²), instead of k.

FIG. 21 is a diagram illustrating an example operation of recording aplurality of objects according to an embodiment.

A first example 2100-1 in which a first speaker 2105 of a plurality ofspeakers utters a voice and a second example 2100-2 in which a secondspeaker 2110 of the plurality of speakers utters a voice will bedescribed with reference to FIG. 21.

It is assumed by way of non-limiting example that video recording isperformed in a situation in which one of a plurality of speakers uttersa voice. The plurality of speakers may include the first speaker 2105and the second speaker 2110. It is assumed that the first speaker 2105stands in front of the camera 110, and the second speaker 2110 stands infront of the center of the front surface portion of the electronicapparatus 100. An object 2115 corresponding to the first speaker 2105may be positioned at the center of image data, and an object 2120corresponding to the second speaker 2110 may be positioned on the rightside of the image data.

In case the plurality of objects 2115 and 2120 corresponding to theplurality of speakers 2105 and 2110, respectively, are identified in theimage data, the electronic apparatus 100 may change sound data inconsideration of position information of each of the plurality ofobjects 2115 and 2120.

In FIG. 21, it is assumed that only one of the plurality of speakers2105 and 2110 utters a voice. The electronic apparatus 100 may identifya speaker who utters a voice among the plurality of speakers 2105 and2110, and change sound data based on position information of theidentified speaker. The electronic apparatus 100 may identify a mouthshape object among various objects included in the image data, and maydetermine movement of the mouth shape object as an uttering motion. Theelectronic apparatus 100 may use an object recognition artificialintelligence module to identify the mouth shape object. The objectrecognition artificial intelligence module may be directly stored in theelectronic apparatus 100, or may be stored in a separate external serverin some implementation examples.

According to the first example 2100-1, in a case where it is identifiedthat the first speaker 2105 utters a voice, the electronic apparatus 100may determine sound ratio information (1:1) based on positioninformation (1:1) of the object 2115 corresponding to the first speaker2105. Further, the electronic apparatus 100 may change sound data basedon the sound ratio information (1:1), and display the changed soundratio information (1:1) in a UI 2125.

According to the second example 2100-2, in a case where it is identifiedthat the second speaker 2110 utters a voice, the electronic apparatus100 may determine sound ratio information (0.5:2) based on positioninformation (2.0.5) of the object 2120 corresponding to the secondspeaker 2110. Further, the electronic apparatus 100 may change sounddata based on the sound ratio information (0.5:2), and display thechanged sound ratio information (0.5:2) in a UI 2130.

FIG. 22 is a flowchart illustrating an example operation illustrated inFIG. 21 according to an embodiment.

Referring to FIG. 22, the electronic apparatus 100 may obtain image data(S2205). The electronic apparatus 100 may obtain first sound data (leftstereo sound data) and second sound data (right stereo sound data)(S2210). The electronic apparatus 100 may determine whether the numberof objects that may make a sound in the image data is plural (S2215).The objects that may make a sound may include a person, an animal, andother various objects. In S2215, not an object that currently makes asound, but an object that is able to make a sound may be identified. Forexample, in case it is identified that the number of person objects istwo, the electronic apparatus 100 may determine that the number ofobjects that may make a sound is plural.

In case the electronic apparatus 100 determines that the number ofobjects that may make a sound is not plural (“N” in operation S2215),the electronic apparatus 100 may change sound data based on positioninformation of one identified object (S2220). The one object may refer,for example, to an object that is determined as currently making asound. The one object does not necessarily refer to an object that doesnot currently make a sound or is not able to make a sound, and sounddata is not changed for such an object.

In case the electronic apparatus 100 determines that the number ofobjects that may make a sound is plural (“Y” in operation S2215), theelectronic apparatus 100 may identify an object that currently makes asound (S2225). The electronic apparatus 100 may obtain positioninformation of the object that currently makes a sound (S2230). Theelectronic apparatus 100 may change sound data based on the positioninformation of the object that currently makes a sound (S2235).

The electronic apparatus 100 may determine whether the object thatcurrently makes a sound is changed (S2240). In case the object thatmakes a sound is not changed (“N” in operation S2240), the electronicapparatus 100 may proceed to the sound data change operation. However,in case the object that makes a sound is changed (“Y”) in operationS2240), the electronic apparatus 100 may repeat S2225, S2230, and S2235to newly obtain position information of the changed object.

FIG. 23 is a diagram illustrating an example operation of recording aplurality of objects according an embodiment.

Referring to FIG. 23, the electronic apparatus 100 may perform videorecording in a situation in which a first speaker 2305 and a secondspeaker 2310 utter a voice at the same time.

The electronic apparatus 100 may obtain first position information of afirst object 2315 corresponding to the first speaker 2305 and secondposition information of a second object 2320 corresponding to the secondspeaker 2310. The electronic apparatus 100 may change sound datacorresponding to the first object 2315 based on the first positioninformation, and change sound data corresponding to the second object2320 based on the second position information.

The electronic apparatus 100 may sort out sound data corresponding toeach object from sound data to obtain the sound data corresponding toeach object. For example, the sound data may include both the voice ofthe first speaker 2305 and the voice of the second speaker 2310. Theelectronic apparatus 100 may extract the voice of the first speaker 2305from the sound data, and may change the extracted voice of the firstspeaker 2305 based on the first position information. Similarly, theelectronic apparatus 100 may extract the voice of the second speaker2310 from the sound data, and may change the extracted voice of thesecond speaker 2310 based on the second position information.

In addition, sound ratio information of each changed sound data may bedisplayed through a UI. Specifically, the sound ratio information (1:1)corresponding to the first speaker 2305 may be provided through a UI2325, and the sound ratio information (0.5:2) corresponding to thesecond speaker 2310 may be provided through a UI 2330.

FIG. 24 is a flowchart illustrating an example operation of theelectronic apparatus illustrated in FIG. 23 according to an embodiment.

Referring to FIG. 24, the electronic apparatus 100 may obtain image data(S2405). The electronic apparatus 100 may obtain first sound data (leftstereo sound data) and second sound data (right stereo sound data)(S2410). S2405 and S2410 may be performed at the same time. For example,the electronic apparatus 100 may obtain the image data, the first sounddata, and the second sound data at the same time.

The electronic apparatus 100 may determine whether the number of objectsthat make a sound is plural (S2415). In case the number of objects thatmake a sound is not plural (“N” in operation S2415), the electronicapparatus 100 may change sound data based on position information of oneobject (S2416).

In case the number of objects that make a sound is plural (“Y” inoperation S2415), the electronic apparatus 100 may obtain sound datacorresponding to each of the plurality of objects from the sound data(S2420). The process of obtaining the sound data corresponding to eachobject may refer, for example, to a process of extracting sound data ofa specific object.

The electronic apparatus 100 may obtain ratio information of the sounddata corresponding to each of the plurality of objects (S2425). Theelectronic apparatus 100 may obtain information on a position where eachof the plurality of objects is displayed on the display (S2430). Theelectronic apparatus 100 may change each sound data based on theposition information of each of the plurality of objects (S2435). Theexample embodiment has been described with reference to FIG. 23, andthus an overlapping description may not be repeated here. The electronicapparatus 100 may display each changed ratio information on the displaythrough a UI.

FIG. 25 is a diagram illustrating an example operation of recording aplurality of objects according to an embodiment.

Referring to FIG. 25, a situation in which a speaker 2505 and a soundoutput device 2510 make a sound at the same time is assumed. A soundmade by the speaker 2505 is described as voice data, and a sound made bythe sound output device 2510 is described as non-voice data.

The electronic apparatus 100 may sort out the voice data and thenon-voice data from sound data. Further, the electronic apparatus 100may perform a sound data change operation on only the voice data withoutperforming the sound data change operation on the non-voice data. Inother words, the electronic apparatus 100 may extract only the voicedata from the sound data. For example, the electronic apparatus 100 mayobtain the voice data by filtering out only a voice frequency from thesound data. The electronic apparatus 100 may store a voice frequencyrange in advance to filter out only the voice frequency, and mayseparate only the voice data from the sound data by identifying only thevoice frequency range.

Performing the sound data change operation on only the voice data may berelated to sound quality. A conversion operation for the non-voice dataand a conversion operation for the voice data may be different in regardto sound quality conversion, and the electronic apparatus 100 mayperform the change operation on only the voice data to output ahigh-quality sound data. The electronic apparatus 100 may change sounddata based on position information of an object 2515 corresponding tothe speaker 2505. In case a setting is made not to change the non-voicedata, the electronic apparatus 100 does not have to consider positioninformation of an object 2520 corresponding to the sound output device2510.

The electronic apparatus 100 may provide sound ratio information of thevoice data through a UI 2525. Further, the electronic apparatus 100 mayprovide sound ratio information of the non-voice data through a UI 2530.

Whether to change only the voice data may be determined depending onuser setting. That is, whether to change both the voice data and thenon-voice data or change only the voice data may be determined dependingon user setting.

FIG. 26 is a flowchart illustrating an example operation of theelectronic apparatus illustrated in FIG. 25 according to an embodiment.

Referring to FIG. 26, the electronic apparatus 100 may obtain image dataand sound data (S2605). The electronic apparatus 100 may obtain positioninformation of an object that generates a sound (S2610).

The electronic apparatus 100 may determine whether voice data isincluded in the sound data (S2615). In case voice data is included inthe sound data (“Y” in operation S2615), the electronic apparatus 100may separate and obtain only the voice data from the sound data (S2620).The electronic apparatus 100 may change the voice data based on positioninformation of an object corresponding to the obtained voice data(S2625).

In case voice data is not included in the sound data (“N” in operationS2615), the electronic apparatus 100 may determine whether to changenon-voice data (S2630). In case a setting for the electronic apparatus100 is made to not change the non-voice data (“N” in operation S2630),the electronic apparatus 100 need not perform the change operation forthe non-voice data. In case a setting for the electronic apparatus 100is made to change the non-voice data (“Y” in operation S2630), theelectronic apparatus 100 may perform the change operation for thenon-voice data based on position information of an object correspondingto the non-voice data (S2635).

FIG. 27 is a flowchart illustrating an example operation in which anobject is guided to be positioned at the center according to anembodiment.

Referring to FIG. 27, the electronic apparatus 100 may obtain image dataand sound data (S2705). The electronic apparatus 100 may obtain positioninformation of an object that generates a sound (S2710).

The electronic apparatus 100 may determine whether the image data is animage obtained through the front camera (S2715).

In case the image data is not an image obtained through the front camera(“N” in operation S2715), the electronic apparatus 100 may change thesound data based on the position information of the object (S2720). Theelectronic apparatus 100 may display, on the display, at least one ofpre-change sound ratio information or post-change sound ratioinformation through a UI (S2725).

In case the image data is an image obtained through the front camera(“Y” in operation S2715), the electronic apparatus 100 may determinewhether a left-right reversal operation is performed on the image data(S2716). For example, the electronic apparatus 100 may determine whetherthe left-right reversal function is applied to the obtained image data.In case the left-right reversal function is not applied to the imagedata (“N” in operation S2716), the electronic apparatus 100 may performthe sound data conversion operation (S2720) and the UI providingoperation (S2725).

In case the left-right reversal function is applied to the image data(“Y” in operation S2716), the electronic apparatus 100 may selectivelyperform the change operation. For example, for the image data appliedwith the left-right reversal function, the electronic apparatus 100 maydetermine whether the object is positioned in a predetermined region inthe image data based on the position information of the object thatgenerates a sound (S2730). In case the object is positioned in thepredetermined region (“Y” in operation S2730), the electronic apparatus100 may perform the sound data conversion operation (S2720) and the UIproviding operation (S2725).

The predetermined region may refer, for example, to a region from acentral point of the image data to a point where the object ispositioned in case sound data having the same volume are obtainedthrough the first microphone 121 and the second microphone 122. Forexample, the predetermined region may refer, for example, to a regionfrom a central vertical axis 2850 to a vertical axis 2855 in FIG. 28B. Adescription thereof will be provided in greater detail below withreference to FIGS. 28A and 28B.

In case the object is not positioned in the predetermined region (“N” inoperation S2730), the electronic apparatus 100 may not change the sounddata (S2735). A reason therefor will be described in greater detailbelow with reference to FIGS. 28A and 28B.

FIGS. 28A and 28B are diagrams illustrating example image data appliedwith the left-right reversal function.

Various examples 2800-1, 2800-2, 2800-3 and 2800-4 in which theleft-right reversal function is applied will be described with referenceto FIGS. 28A and 28B.

FIGS. 28A and 28B commonly illustrate a speaker 2805, an object 2810corresponding to the speaker 2805, and a UI 2815 that displays soundratio information.

According to the first example 2800-1, it is assumed that the speaker2805 faces the camera 110 on the right side. The image data applied withthe left-right reversal function to display the object 2810corresponding to the speaker 2805 in a left region may be generated.Because the speaker 2805 utters a voice near the first microphone 121,the volume of first sound data obtained through the first microphone 121may be higher than that of second sound data obtained through the secondmicrophone 122. A person who watches a video may feel that the imagedata and the sound data are balanced. Therefore, the electronicapparatus 100 may determine that the sound data change operation is notnecessary in such a situation.

However, according to the second example 2800-2, in case the speaker2805 is positioned in front of the camera 110, the electronic apparatus100 may determine that the sound data change operation is necessary. Incase the speaker 2805 is positioned in front of the camera 110, theobject 2810 corresponding to the speaker 2805 may be positioned at thecenter of the image data. The image data may have the central verticalaxis 2850. The electronic apparatus 100 may determine that the sounddata change is not necessary in case the object 2810 is positioned onthe left of the central vertical axis 2850 in the image data. Further,the electronic apparatus 100 may determine that the sound data change isnecessary in case the object 2810 is positioned on the central verticalaxis 2850 in the image data displayed on the electronic apparatus 100.

According to the third example 2800-3, in case the speaker 2805 ispositioned in front of the front surface portion of the electronicapparatus 100, the object 2810 corresponding to the speaker 2805 may bepositioned on the right of the central vertical axis 2850 in the imagedata. In a situation in which the speaker 2805 stands in front of thecenter of the front surface portion of the electronic apparatus 100, thevolumes of sound data received through the first microphone 121 and thesecond microphone 122, respectively, may be the same as each other.Further, in case the volumes of sound data received through themicrophones, respectively, are the same as each other, the vertical axis2855 of a position where the object 2810 is displayed may be obtained.The electronic apparatus 100 may determine that the sound data change isnecessary in case the object 2810 is positioned in a region between thecentral vertical axis 2850 and the vertical axis 2855 in the image datadisplayed on the electronic apparatus 100.

According to the fourth example 2800-4, the electronic apparatus 100 maydetermine that the sound data change is not necessary in case the object2810 is positioned in a region on the right of the vertical line 2855 inthe image data.

In summary, the electronic apparatus 100 may determine that the sounddata change is necessary in case the object 2810 is positioned in theregion between the central vertical axis 2850 and the vertical axis 2855in the image data displayed on the electronic apparatus 100, and maydetermine that the sound data change is not necessary in case the object2810 is positioned in a region other than the region between the centralvertical axis 2850 and the vertical axis 2855 in the image data.

In case the object 2810 is positioned in the region between the centralvertical axis 2850 and the vertical axis 2855 (e.g., region 2860), aperson who watches a video may feel that the image data and the sounddata do not match (unnatural) each other, and in case the object 2810 ispositioned in the region other than the region between the centralvertical axis 2850 and the vertical axis 2855, a person who watches avideo may feel that the image data and the sound data match each other.

The embodiment illustrated in FIGS. 27, 28A and 28B describes an exampleoperation of identifying whether the left-right reversal function isapplied and identifying the region in which the object is positioned inthe image data. Such an operation may decrease a processing load on amemory as compared to a case of changing sound data in all videos.Further, the electronic apparatus 100 illustrated in FIGS. 27, 28A and28B does not perform the sound data conversion operation in a specificregion, and thus it is possible to perform the control operationefficiently in terms of processing speed and power management.

FIG. 29 is a flowchart illustrating an example method of controlling theelectronic apparatus according to an embodiment.

Referring to FIG. 29, the electronic apparatus 100 according to anexample embodiment may obtain image data through the camera 110, obtainfirst sound data through the first microphone 121, and obtain secondsound data through the second microphone 122 (S2905).

The electronic apparatus 100 may identify an object corresponding to thefirst sound data and the second sound data in the obtained image data(S2910).

The electronic apparatus 100 may obtain position information of theidentified object from the image data (S2915).

The electronic apparatus 100 may change volume information of at leastone of the first sound data or the second sound data based on theobtained position information (S2920).

The first microphone 121 and the second microphone 122 are spaced apartfrom each other, and a first distance between a position of the camera110 and a position of the first microphone 121, and a second distancebetween the position of the camera 110 and a position of the secondmicrophone 122 may be different from each other.

In the identifying of the object (S2910), a subject or object thatutters the first sound data and the second sound data may be identifiedas the object corresponding to the first sound data and the second sounddata in the image data.

The method of controlling the electronic apparatus may further includeobtaining sound ratio information between the volume information of thefirst sound data and the volume information of the second sound data,and in the changing of the volume information (S2920), the volumeinformation of at least one of the first sound data or the second sounddata may be changed based on the sound ratio information and theposition information of the identified object.

The method may further include obtaining weighted value information forchanging the volume information of at least one of the first sound dataor the second sound data based on the sound ratio information and theposition information of the identified object, and in the changing ofthe volume information (S2920), the volume information of at least oneof the first sound data or the second sound data may be changed based onthe obtained weighted value information.

In the obtaining of the weighted value information, the weighted valueinformation for changing the volume information of at least one of thefirst sound data or the second sound data may be obtained based on thesound ratio information, and distance ratio information between a firstdistance from one side of the image data to the identified object, and asecond distance from the other side of the image data to the identifiedobject, and the other side of the image data may be opposite to the oneside of the image data.

In the changing of the volume information (S2920), the weighted valueinformation may be obtained by multiplying the distance ratioinformation and the sound ratio information, and the volume informationof the first sound data or the second sound data may be changed bymultiplying the volume of the first sound data or the second sound databy the weighted value information.

In the changing of the volume information (S2920), the volumeinformation may be changed by multiplying the volume of one of the firstsound data or the second sound data by the weighted value information,the one sound data having a higher volume.

The method may further include displaying a UI including the changedvolume information of the first sound data or the second sound data.

In the changing of the volume information (S2920), distance informationbetween a user corresponding to the identified object and the electronicapparatus 100 may be obtained based on the image data, and the volumeinformation of at least one of the first sound data or the second sounddata may be changed based on the obtained position information based onthe obtained distance information being less than a threshold distance.

The method may further include obtaining, based on first and secondobjects being identified in the image data, first position informationof the identified first object and second position information of theidentified second object from the image data, and in the changing of thevolume information (S2920), volume information of at least one of firstsound data or second sound data corresponding to the first object may bechanged based on the first position information, and volume informationof at least one of third sound data or fourth sound data correspondingto the second object may be changed based on the second positioninformation.

The method of the electronic apparatus as illustrated in FIG. 29 may beperformed on the electronic apparatus having the configuration of FIG. 2or 3, and may also be performed on an electronic apparatus having otherconfigurations.

The methods according to the various example embodiments of thedisclosure described above may be implemented in a form of anapplication that may be installed in the existing electronic apparatus.

The methods according to the various example embodiments of thedisclosure described above may be implemented by performing a softwareupgrade or a hardware upgrade with respect to the existing electronicapparatus.

Further, the various example embodiments of the disclosure describedabove may be executed through an embedded server provided in theelectronic apparatus, or an external server of at least one of theelectronic apparatus or the display device.

According to an embodiment of the disclosure, the various exampleembodiments described above may be implemented by software includinginstructions stored in a machine-readable storage medium (for example, acomputer-readable storage medium). The machine may be an apparatus thatmay invoke a stored instruction from a storage medium and may beoperated according to the invoked instruction. The machine may includethe electronic apparatus according to the disclosed embodiments. In casean instruction is executed by the processor, the processor may directlyperform a function corresponding to the instruction or other componentsmay perform the function corresponding to the instruction under acontrol of the processor. The instruction may a code made by a compileror a code executable by an interpreter. The machine-readable storagemedium may be provided in a form of a non-transitory storage medium. The“non-transitory” storage medium is tangible an may not include a signal,and does not distinguish whether data is semi-permanently or temporarilystored on the storage medium.

In addition, according to an embodiment of the disclosure, the methodsaccording to the various embodiments described above may be included andprovided in a computer program product. The computer program product maybe traded as a product between a seller and a purchaser. The computerprogram product may be distributed in a form of a storage medium (forexample, a compact disc read only memory (CD-ROM)) that may be read bythe machine or online through an application store (for example,PlayStore™). In case of the online distribution, at least a part of thecomputer program product may be at least temporarily stored in a storagemedium such as a memory of a server of a manufacturer, a server of anapplication store, or a relay server or be temporarily created.

In addition, each of components (for example, modules or programs)according to the various embodiments described above may include asingle entity or a plurality of entities, and some of the correspondingsub-components described above may be omitted or other sub-componentsmay be further included in the various embodiments. Alternatively oradditionally, some of the components (for example, the modules or theprograms) may be integrated into one entity, and may perform functionsperformed by the respective corresponding components before beingintegrated in the same or similar manner. Operations performed by themodules, the programs, or other components according to the variousembodiments may be executed in a sequential manner, a parallel manner,an iterative manner, or a heuristic manner, at least some of theoperations may be performed in a different order or be omitted, or otheroperations may be added.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill be further understood by one of ordinary skill in the art thatvarious changes in form and detail may be made without departing fromthe true spirit and full scope of the disclosure, including the appendedclaims and their equivalents.

What is claimed is:
 1. An electronic apparatus comprising: a camera; afirst microphone; a second microphone; and a processor configured to:control the electronic apparatus to obtain first sound data through thefirst microphone and obtain second sound data through the secondmicrophone, obtain image data through the camera, identify an objectcorresponding to the first sound data and the second sound data among atleast one of objects in the obtained image data, obtain positioninformation of the identified object from the image data, and changevolume information of at least one of the first sound data obtainedthrough the first microphone or the second sound data obtained throughthe second microphone based on the obtained position information.
 2. Theelectronic apparatus as claimed in claim 1, wherein the first microphoneand the second microphone are spaced apart from each other, and a firstdistance between a position of the camera and a position of the firstmicrophone, and a second distance between the position of the camera anda position of the second microphone are different from each other. 3.The electronic apparatus as claimed in claim 1, wherein the processor isconfigured to identify, as the object corresponding to the first sounddata and the second sound data, a subject that utters the first sounddata and the second sound data in the image data.
 4. The electronicapparatus as claimed in claim 1, wherein the processor is configured to:obtain sound ratio information between the volume information of thefirst sound data and the volume information of the second sound data,and change the volume information of at least one of the first sounddata or the second sound data based on the sound ratio information andthe position information of the identified object.
 5. The electronicapparatus as claimed in claim 4, wherein the processor is configured to:obtain weighted value information for changing the volume information ofat least one of the first sound data or the second sound data based onthe sound ratio information and the position information of theidentified object, and change the volume information of at least one ofthe first sound data or the second sound data based on the obtainedweighted value information.
 6. The electronic apparatus as claimed inclaim 5, wherein the processor is configured to obtain the weightedvalue information for changing the volume information of at least one ofthe first sound data or the second sound data based on the sound ratioinformation, and distance ratio information between a first distancefrom one side of the image data to the identified object, and a seconddistance from an other side of the image data to the identified object,and wherein the other side of the image data is opposite to the one sideof the image data.
 7. The electronic apparatus as claimed in claim 6,wherein the processor is configured to: obtain the weighted valueinformation by multiplying the distance ratio information and the soundratio information, and change the volume information of the first sounddata or the second sound data by multiplying the volume information ofthe first sound data or the second sound data by the weighted valueinformation.
 8. The electronic apparatus as claimed in claim 1, furthercomprising a display, wherein the processor is configured to control thedisplay to display a user interface (UI) including the changed volumeinformation of the first sound data or the second sound data.
 9. Theelectronic apparatus as claimed in claim 1, wherein the processor isconfigured to: obtain distance information between a subjectcorresponding to the identified object and the electronic apparatusbased on the image data, and change the volume information of at leastone of the first sound data or the second sound data based on theobtained position information based on the obtained distance informationbeing less than a threshold distance.
 10. The electronic apparatus asclaimed in claim 1, wherein the processor is configured to: obtain,based on first and second objects being identified in the image data,first position information of the identified first object and secondposition information of the identified second object from the imagedata, change volume information of at least one of first sound data orsecond sound data corresponding to the first object based on the firstposition information, and change volume information of at least one ofthird sound data or fourth sound data corresponding to the second objectbased on the second position information.
 11. A method of controlling anelectronic apparatus, the method comprising: obtaining image datathrough a camera, obtaining first sound data through a first microphone,and obtaining second sound data through a second microphone; identifyingan object corresponding to the first sound data and the second sounddata among at least one of objects in the obtained image data; obtainingposition information of the identified object from the image data; andchanging volume information of at least one of the first sound dataobtained through the first microphone or the second sound data obtainedthrough the second microphone based on the obtained positioninformation.
 12. The method as claimed in claim 11, wherein the firstmicrophone and the second microphone are spaced apart from each other,and a first distance between a position of the camera and a position ofthe first microphone, and a second distance between the position of thecamera and a position of the second microphone are different from eachother.
 13. The method as claimed in claim 11, wherein in the identifyingof the object, a subject that utters the first sound data and the secondsound data is identified as the object corresponding to the first sounddata and the second sound data in the image data.
 14. The method asclaimed in claim 11, further comprising obtaining sound ratioinformation between the volume information of the first sound data andthe volume information of the second sound data, wherein in the changingof the volume information, the volume information of at least one of thefirst sound data or the second sound data is changed based on the soundratio information and the position information of the identified object.15. The method as claimed in claim 14, further comprising obtainingweighted value information for changing the volume information of atleast one of the first sound data or the second sound data based on thesound ratio information and the position information of the identifiedobject, wherein in the changing of the volume information, the volumeinformation of at least one of the first sound data or the second sounddata is changed based on the obtained weighted value information. 16.The method as claimed in claim 15, wherein in the obtaining of theweighted value information, the weighted value information for changingthe volume information of at least one of the first sound data or thesecond sound data is obtained based on the sound ratio information, anddistance ratio information between a first distance from one side of theimage data to the identified object, and a second distance from an otherside of the image data to the identified object, and wherein the otherside of the image data is opposite to the one side of the image data.17. The method as claimed in claim 16, wherein in the changing of thevolume information, the weighted value information is obtained bymultiplying the distance ratio information and the sound ratioinformation, and the volume information of the first sound data or thesecond sound data is changed by multiplying the volume information ofthe first sound data or the second sound data by the weighted valueinformation.
 18. The method as claimed in claim 17, further comprisingdisplaying a UI including the changed volume information of the firstsound data or the second sound data.
 19. The method as claimed in claim11, wherein in the changing of the volume information, distanceinformation between a subject corresponding to the identified object andthe electronic apparatus is obtained based on the image data, and thevolume information of at least one of the first sound data or the secondsound data is changed based on the obtained position information basedon the obtained distance information being less than a thresholddistance.
 20. The method as claimed in claim 11, further comprisingobtaining, based on first and second objects being identified in theimage data, first position information of the identified first objectand second position information of the identified second object from theimage data, wherein in the changing of the volume information, volumeinformation of at least one of first sound data or second sound datacorresponding to the first object is changed based on the first positioninformation, and volume information of at least one of third sound dataor fourth sound data corresponding to the second object is changed basedon the second position information.